Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spectral Gap of Metropolis Algorithms for Non-smooth Distributions under Isoperimetry

Shuigen Liu, Xin T. Tong

TL;DR

The paper develops explicit spectral-gap bounds for Random--Walk Metropolis and Metropolis--adjusted Langevin algorithms when targeting non-smooth distributions, by extending the isoperimetric framework with a new close-coupling argument. It shows that, under a smooth-plus-Lipschitz log-density log π = f + g with f smooth and g Lipschitz, RWM and MALA exhibit gaps scaling with the step size and problem conditioning, including additional non-smooth-induced terms; the results also extend to targets satisfying Poincaré or log-Sobolev inequalities via recent work. A notable additional contribution is a uniform lower bound on MALA’s acceptance rate under strong log-concavity with concavity on the smooth part, enabling a robust close-coupling analysis. Numerical experiments on Bayesian Lasso and logistic regression with Laplace priors validate the theoretical scaling and illustrate practical implications for step-size choices in non-smooth settings.

Abstract

Metropolis algorithms are classical tools for sampling from target distributions, with broad applications in statistics and scientific computing. Their convergence speed is governed by the spectral gap of the associated Markov operator. Recently, Andrieu et al. (2024) derived the first explicit bounds for the spectral gap of Random-Walk Metropolis when the target distribution is smooth and strongly log-concave. However, existing literature rarely discuss non-smooth targets. In this work, we derive explicit spectral gap bounds for the Random-Walk Metropolis and Metropolis-adjusted Langevin algorithms over a broad class of non-smooth distributions. Moreover, combining our analysis with a recent result in Goyal et al. (2025), we extend these bounds to targets satisfying a Poincare or log-Sobolev inequality, beyond the strongly log-concave setting. Our theoretical results are further supported by numerical experiments.

Spectral Gap of Metropolis Algorithms for Non-smooth Distributions under Isoperimetry

TL;DR

The paper develops explicit spectral-gap bounds for Random--Walk Metropolis and Metropolis--adjusted Langevin algorithms when targeting non-smooth distributions, by extending the isoperimetric framework with a new close-coupling argument. It shows that, under a smooth-plus-Lipschitz log-density log π = f + g with f smooth and g Lipschitz, RWM and MALA exhibit gaps scaling with the step size and problem conditioning, including additional non-smooth-induced terms; the results also extend to targets satisfying Poincaré or log-Sobolev inequalities via recent work. A notable additional contribution is a uniform lower bound on MALA’s acceptance rate under strong log-concavity with concavity on the smooth part, enabling a robust close-coupling analysis. Numerical experiments on Bayesian Lasso and logistic regression with Laplace priors validate the theoretical scaling and illustrate practical implications for step-size choices in non-smooth settings.

Abstract

Metropolis algorithms are classical tools for sampling from target distributions, with broad applications in statistics and scientific computing. Their convergence speed is governed by the spectral gap of the associated Markov operator. Recently, Andrieu et al. (2024) derived the first explicit bounds for the spectral gap of Random-Walk Metropolis when the target distribution is smooth and strongly log-concave. However, existing literature rarely discuss non-smooth targets. In this work, we derive explicit spectral gap bounds for the Random-Walk Metropolis and Metropolis-adjusted Langevin algorithms over a broad class of non-smooth distributions. Moreover, combining our analysis with a recent result in Goyal et al. (2025), we extend these bounds to targets satisfying a Poincare or log-Sobolev inequality, beyond the strongly log-concave setting. Our theoretical results are further supported by numerical experiments.
Paper Structure (28 sections, 11 theorems, 92 equations, 4 figures)

This paper contains 28 sections, 11 theorems, 92 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Notations
  3. Metropolis algorithms and non-smooth distributions
  4. Metropolis algorithms
  5. Random--Walk Metropolis
  6. Metropolis--adjusted Langevin algorithm
  7. Non-smooth distributions
  8. Metropolis algorithms for non-smooth distributions
  9. Isoperimetric approach to the spectral gap
  10. Spectral gap
  11. Isoperimetric approach
  12. Three-set isoperimetric inequality
  13. Spectral gap of Metropolis algorithms for non-smooth distributions
  14. Spectral gap of Random--Walk Metropolis
  15. Close coupling condition
  16. ...and 13 more sections

Key Result

Theorem 3.1

Let $\mathsf{P}$ be a reversible Markov kernel. Then it holds where $\kappa \ge 1$ is a universal constant.

Figures (4)

  • Figure 1: Estimated square root of the spectral gap (left) and average acceptance rate (right) of RWM for Bayesian Lasso model under different step size $h$ and regularization parameter $\lambda$
  • Figure 2: Estimated spectral gap (left) and average acceptance rate (right) of MALA for Bayesian Lasso model under different step size $h$ and regularization parameter $\lambda$
  • Figure 3: Estimated square root of the spectral gap (left) and average acceptance rate (right) of RWM for Bayesian Logistic regression model under different step size $h$ and regularization parameter $\lambda$
  • Figure 4: Estimated spectral gap (left) and average acceptance rate (right) of MALA for Bayesian Logistic regression model under different step size $h$ and regularization parameter $\lambda$

Theorems & Definitions (27)

  • Remark 2.1
  • Definition 3.1
  • Theorem 3.1: MR930082
  • Proposition 3.2
  • Definition 3.2
  • Proposition 3.3: 24-AAP2058, Lemma 27
  • Proposition 3.4: goyal2025mixing, Propositions 4.1 and 4.3
  • Remark 3.1
  • Lemma 4.1
  • Remark 4.1
  • ...and 17 more