Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Sampling on Riemannian Manifolds via Langevin MCMC

Xiang Cheng, Jingzhao Zhang, Suvrit Sra

TL;DR

This work develops nonasymptotic, geometry-aware Langevin MCMC methods for sampling from Gibbs measures on Riemannian manifolds. By establishing a discretization error bound of order $O(\delta^3)$ per step for geometric Euler–Maruyama and proving Wasserstein contraction via Kendall–Cranston coupling under manifold distant-dissipativity, it shows that the discrete Langevin iterates approximate the target distribution in $W_1$ after $\tilde{O}(\varepsilon^{-2})$ steps, matching the Euclidean rate. The framework accommodates nonconvex $h$ and negative Ricci curvature, and extends to stochastic-gradient variants under curvature-dimension conditions $CD(\cdot,\infty)$, with similar iteration complexity. Practical discretizations using exponential maps (and retrac-tion variants) are discussed, and a sphere example illustrates the assumptions in a concrete setting. Overall, the results unify Euclidean Langevin scaling with intrinsic manifold geometry to enable efficient sampling on manifolds.

Abstract

We study the task of efficiently sampling from a Gibbs distribution $d π^* = e^{-h} d {vol}_g$ over a Riemannian manifold $M$ via (geometric) Langevin MCMC; this algorithm involves computing exponential maps in random Gaussian directions and is efficiently implementable in practice. The key to our analysis of Langevin MCMC is a bound on the discretization error of the geometric Euler-Murayama scheme, assuming $\nabla h$ is Lipschitz and $M$ has bounded sectional curvature. Our error bound matches the error of Euclidean Euler-Murayama in terms of its stepsize dependence. Combined with a contraction guarantee for the geometric Langevin Diffusion under Kendall-Cranston coupling, we prove that the Langevin MCMC iterates lie within $ε$-Wasserstein distance of $π^*$ after $\tilde{O}(ε^{-2})$ steps, which matches the iteration complexity for Euclidean Langevin MCMC. Our results apply in general settings where $h$ can be nonconvex and $M$ can have negative Ricci curvature. Under additional assumptions that the Riemannian curvature tensor has bounded derivatives, and that $π^*$ satisfies a $CD(\cdot,\infty)$ condition, we analyze the stochastic gradient version of Langevin MCMC, and bound its iteration complexity by $\tilde{O}(ε^{-2})$ as well.

Efficient Sampling on Riemannian Manifolds via Langevin MCMC

TL;DR

This work develops nonasymptotic, geometry-aware Langevin MCMC methods for sampling from Gibbs measures on Riemannian manifolds. By establishing a discretization error bound of order per step for geometric Euler–Maruyama and proving Wasserstein contraction via Kendall–Cranston coupling under manifold distant-dissipativity, it shows that the discrete Langevin iterates approximate the target distribution in after steps, matching the Euclidean rate. The framework accommodates nonconvex and negative Ricci curvature, and extends to stochastic-gradient variants under curvature-dimension conditions , with similar iteration complexity. Practical discretizations using exponential maps (and retrac-tion variants) are discussed, and a sphere example illustrates the assumptions in a concrete setting. Overall, the results unify Euclidean Langevin scaling with intrinsic manifold geometry to enable efficient sampling on manifolds.

Abstract

We study the task of efficiently sampling from a Gibbs distribution over a Riemannian manifold via (geometric) Langevin MCMC; this algorithm involves computing exponential maps in random Gaussian directions and is efficiently implementable in practice. The key to our analysis of Langevin MCMC is a bound on the discretization error of the geometric Euler-Murayama scheme, assuming is Lipschitz and has bounded sectional curvature. Our error bound matches the error of Euclidean Euler-Murayama in terms of its stepsize dependence. Combined with a contraction guarantee for the geometric Langevin Diffusion under Kendall-Cranston coupling, we prove that the Langevin MCMC iterates lie within -Wasserstein distance of after steps, which matches the iteration complexity for Euclidean Langevin MCMC. Our results apply in general settings where can be nonconvex and can have negative Ricci curvature. Under additional assumptions that the Riemannian curvature tensor has bounded derivatives, and that satisfies a condition, we analyze the stochastic gradient version of Langevin MCMC, and bound its iteration complexity by as well.
Paper Structure (34 sections, 41 theorems, 394 equations, 1 figure)

This paper contains 34 sections, 41 theorems, 394 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Overview of Main Contributions
  3. Preliminaries: Key Assumptions and Notation
  4. An illustrative example: sampling from a sphere
  5. Bounding the Discretization Error in Geometric Euler-Murayama
  6. Langevin MCMC on Riemannian Manifolds
  7. Proof Sketch of Theorem \ref{['t:langevin_mcmc']}
  8. Stochastic Gradient Langevin MCMC
  9. Acknowledgements
  10. Manifold SDE
  11. SDE Construction
  12. Existence of Limit
  13. Discretization Error of Euler Murayama
  14. Proof of Theorem \ref{['t:langevin_mcmc']}
  15. Proof of Theorem \ref{['t:SGLD']}
  16. ...and 19 more sections

Key Result

Lemma 1

Let $x(t)$ denote the solution to e:intro_sde initialized at some $x(0)$. Let $x^0(t)$ denote one step geometric Euler Murayama discretization: $x^0(t) := \mathop{\mathrm{Exp}}\nolimits_{x(0)}{(t \beta(x(0)) + \sqrt{t} \zeta)}$, where $\zeta \sim \mathcal{N}_{x(0)}(0,I)$. Under Assumptions ass:beta_ where $O()$ hides polynomial dependence on the Lipschitz constant $L_\beta'$, the sectional curvatu

Figures (1)

  • Figure 1: An illustration of the coupling between paths for $x^i$ and $x^{i+1}$ when $\beta = 0$ over the time interval $[k\delta^i, (k+1)\delta^i]$.

Theorems & Definitions (83)

  • Lemma 1: Informal version of Lemma \ref{['l:discretization-approximation-lipschitz-derivative']}
  • Remark 1
  • Lemma 2
  • Theorem 1: Convergence of Langevin MCMC on Riemannian Manifold
  • Lemma 3
  • Theorem 2: Convergence of SGLD on Riemannian Manifold
  • proof : Proof of Lemma \ref{['l:Phi_is_diffusion']}
  • Lemma 4
  • proof
  • Lemma 5
  • ...and 73 more