Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Performance Of The Unadjusted Langevin Algorithm Without Smoothness Assumptions

Tim Johnston, Iosif Lytras, Nikolaos Makras, Sotirios Sabanis

TL;DR

The paper tackles the challenge of sampling from non-smooth, non-log-concave densities by introducing the Subgradient Unadjusted Langevin Algorithm (SG-ULA), an explicit Euler–Maruyama discretization that uses subgradients of a semi-convex potential and avoids smoothing techniques. It proves non-asymptotic Wasserstein convergence with explicit rates in terms of the step size, including lambda^{1/4} savings in W1 and lambda^{1/8} in W2 under stronger assumptions, and provides an excess-risk optimization bound tying sampling accuracy to optimization performance. Through numerical experiments on mixtures of Gaussians with Laplacian priors and SCAD penalties, SG-ULA demonstrates robust multimodal sampling and improved excess-risk performance compared to smoothing-based approaches. These results extend Langevin-based methods to broader, non-smooth, non-convex settings with explicit, dimension-sensitive guarantees, offering practical benefits for Bayesian inference and regularized inverse problems.

Abstract

In this article, we study the problem of sampling from distributions whose densities are not necessarily smooth nor logconcave. We propose a simple Langevin-based algorithm that does not rely on popular but computationally challenging techniques, such as the Moreau-Yosida envelope or Gaussian smoothing, and show consequently that the performance of samplers like ULA does not necessarily degenerate arbitrarily with low regularity. In particular, we show that the Lipschitz or Hölder continuity assumption can be replaced by a geometric one-sided Lipschitz condition that allows even for discontinuous log-gradients. We derive non-asymptotic guarantees for the convergence of the algorithm to the target distribution in Wasserstein distances. Non-asymptotic bounds are also provided for the performance of the algorithm as an optimizer, specifically for the solution of associated excess risk optimization problems.

The Performance Of The Unadjusted Langevin Algorithm Without Smoothness Assumptions

TL;DR

The paper tackles the challenge of sampling from non-smooth, non-log-concave densities by introducing the Subgradient Unadjusted Langevin Algorithm (SG-ULA), an explicit Euler–Maruyama discretization that uses subgradients of a semi-convex potential and avoids smoothing techniques. It proves non-asymptotic Wasserstein convergence with explicit rates in terms of the step size, including lambda^{1/4} savings in W1 and lambda^{1/8} in W2 under stronger assumptions, and provides an excess-risk optimization bound tying sampling accuracy to optimization performance. Through numerical experiments on mixtures of Gaussians with Laplacian priors and SCAD penalties, SG-ULA demonstrates robust multimodal sampling and improved excess-risk performance compared to smoothing-based approaches. These results extend Langevin-based methods to broader, non-smooth, non-convex settings with explicit, dimension-sensitive guarantees, offering practical benefits for Bayesian inference and regularized inverse problems.

Abstract

In this article, we study the problem of sampling from distributions whose densities are not necessarily smooth nor logconcave. We propose a simple Langevin-based algorithm that does not rely on popular but computationally challenging techniques, such as the Moreau-Yosida envelope or Gaussian smoothing, and show consequently that the performance of samplers like ULA does not necessarily degenerate arbitrarily with low regularity. In particular, we show that the Lipschitz or Hölder continuity assumption can be replaced by a geometric one-sided Lipschitz condition that allows even for discontinuous log-gradients. We derive non-asymptotic guarantees for the convergence of the algorithm to the target distribution in Wasserstein distances. Non-asymptotic bounds are also provided for the performance of the algorithm as an optimizer, specifically for the solution of associated excess risk optimization problems.

Paper Structure

This paper contains 30 sections, 25 theorems, 105 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Results for potentials with Lipschitz-smooth gradients
  4. Results for non-Lipschitz smooth gradients
  5. Results for non-smooth potentials
  6. Euler Scheme Approximations
  7. Summary of contributions and comparison with literature
  8. Notation
  9. The Non-Convex Setting
  10. Subdifferentiability for non-smooth functions - subgradients
  11. Assumptions
  12. Main Results
  13. Theoretical Guarantees
  14. Overview of proof techniques
  15. Examples and Numerical Experiments
  16. ...and 15 more sections

Key Result

Lemma 1

Let $u$ be a semi-convex function. Then, $u$ is locally Lipschitz continuous, the sets $\partial u(x)$ are non-empty, compact, and $p \in$$\partial u(x)$, if and only if

Figures (4)

  • Figure 1: Comparison of SGULA and MYULA on a two dimensional mixture of Gaussians target with a Laplace prior. The top row corresponds to case $K=3$, and the bottom row to $K=5$.
  • Figure 2: Distribution of median relative model errors (MRME %) over 100 Monte Carlo replications for robust regression using SGULA
  • Figure 3: SGULA applied to a two-dimensional Gaussian mixture with Laplace prior, for varying stepsizes.
  • Figure 4: SGULA applied to a two-dimensional Gaussian mixture with Laplace prior, for varying inverse temperature parameters.

Theorems & Definitions (56)

  • Definition 1
  • Lemma 1: alberti1992singularities, Proposition 2.1, adapted
  • Corollary 1
  • Lemma 2
  • proof
  • Theorem 1
  • proof
  • Corollary 2
  • Corollary 3
  • Remark 1
  • ...and 46 more