Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Shifted Composition IV: Toward Ballistic Acceleration for Log-Concave Sampling

Jason M. Altschuler, Sinho Chewi, Matthew S. Zhang

TL;DR

The paper tackles the open problem of achieving ballistic acceleration for log-concave sampling via discretized underdamped Langevin dynamics. It develops a coupling-based KL framework that handles degenerate diffusion and discretization, using an auxiliary interpolating process and shifted Girsanov with time-varying twisted coordinates to control errors without requiring contractivity. The authors establish continuous-time Harnack-type reverse transport inequalities for ULD, introduce a KL local error framework for discretizations, and apply these results to sampling algorithms, proving the first ballistic-acceleration result for RM--ULMC with a sublinear in the condition number complexity, along with a dimension-aware bound via space-time Poincaré inequalities. The work connects nonreversible dynamics, entropic hypocoercivity, and discretization error control to provide practical, dimension-sensitive guarantees for high-dimensional sampling and warm-start scenarios. Overall, it advances both the theory and practice of accelerated sampling for log-concave targets by bridging continuous-time acceleration with discretized algorithms."

Abstract

Acceleration is a celebrated cornerstone of convex optimization, enabling gradient-based algorithms to converge sublinearly in the condition number. A major open question is whether an analogous acceleration phenomenon is possible for log-concave sampling. Underdamped Langevin dynamics (ULD) has long been conjectured to be the natural candidate for acceleration, but a central challenge is that its degeneracy necessitates the development of new analysis approaches, e.g., the theory of hypocoercivity. Although recent breakthroughs established ballistic acceleration for the (continuous-time) ULD diffusion via space-time Poincare inequalities, (discrete-time) algorithmic results remain entirely open: the discretization error of existing analysis techniques dominates any continuous-time acceleration. In this paper, we give a new coupling-based local error framework for analyzing ULD and its numerical discretizations in KL divergence. This extends the framework in Shifted Composition III from uniformly elliptic diffusions to degenerate diffusions, and shares its virtues: the framework is user-friendly, applies to sophisticated discretization schemes, and does not require contractivity. Applying this framework to the randomized midpoint discretization of ULD establishes the first ballistic acceleration result for log-concave sampling (i.e., sublinear dependence on the condition number). Along the way, we also obtain the first $d^{1/3}$ iteration complexity guarantee for sampling to constant total variation error in dimension $d$.

Shifted Composition IV: Toward Ballistic Acceleration for Log-Concave Sampling

TL;DR

The paper tackles the open problem of achieving ballistic acceleration for log-concave sampling via discretized underdamped Langevin dynamics. It develops a coupling-based KL framework that handles degenerate diffusion and discretization, using an auxiliary interpolating process and shifted Girsanov with time-varying twisted coordinates to control errors without requiring contractivity. The authors establish continuous-time Harnack-type reverse transport inequalities for ULD, introduce a KL local error framework for discretizations, and apply these results to sampling algorithms, proving the first ballistic-acceleration result for RM--ULMC with a sublinear in the condition number complexity, along with a dimension-aware bound via space-time Poincaré inequalities. The work connects nonreversible dynamics, entropic hypocoercivity, and discretization error control to provide practical, dimension-sensitive guarantees for high-dimensional sampling and warm-start scenarios. Overall, it advances both the theory and practice of accelerated sampling for log-concave targets by bridging continuous-time acceleration with discretized algorithms."

Abstract

Acceleration is a celebrated cornerstone of convex optimization, enabling gradient-based algorithms to converge sublinearly in the condition number. A major open question is whether an analogous acceleration phenomenon is possible for log-concave sampling. Underdamped Langevin dynamics (ULD) has long been conjectured to be the natural candidate for acceleration, but a central challenge is that its degeneracy necessitates the development of new analysis approaches, e.g., the theory of hypocoercivity. Although recent breakthroughs established ballistic acceleration for the (continuous-time) ULD diffusion via space-time Poincare inequalities, (discrete-time) algorithmic results remain entirely open: the discretization error of existing analysis techniques dominates any continuous-time acceleration. In this paper, we give a new coupling-based local error framework for analyzing ULD and its numerical discretizations in KL divergence. This extends the framework in Shifted Composition III from uniformly elliptic diffusions to degenerate diffusions, and shares its virtues: the framework is user-friendly, applies to sophisticated discretization schemes, and does not require contractivity. Applying this framework to the randomized midpoint discretization of ULD establishes the first ballistic acceleration result for log-concave sampling (i.e., sublinear dependence on the condition number). Along the way, we also obtain the first iteration complexity guarantee for sampling to constant total variation error in dimension .

Paper Structure

This paper contains 55 sections, 42 theorems, 194 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Contribution
  3. Harnack inequalities for the underdamped Langevin diffusion
  4. KL local error framework for underdamped Langevin discretizations
  5. Applications to sampling
  6. Techniques
  7. Related work
  8. Preliminaries
  9. Harnack inequalities for the underdamped Langevin diffusion
  10. Main result
  11. Construction of auxiliary process
  12. Warm up: integrated Brownian motion
  13. Distance bound for auxiliary process
  14. Distance decay identity
  15. Contraction
  16. ...and 40 more sections

Key Result

Theorem 1.1

Let $\pi$ be a strongly log-concave and log-smooth measure over $\mathbb{R}^d$ with condition number $\kappa$. The randomized midpoint discretization of the underdamped Langevin diffusion eq:rm-ulmc, when initialized from a measure $\mu_0$ with $\log \chi^2(\mu_0 \mathbin{\|} \pi) = \widetilde{O}(d)

Figures (1)

  • Figure 1: Shift schedules $\{\eta_t^\mathsf{p}\}_{t\in [0,T]}$ from Definition \ref{['def:ct-shifts']}, for different convexity parameters $\alpha$, and with values of moderate $T = 3$ (left) and small $T=0.1$ (right). These are all in the high friction setting.

Theorems & Definitions (86)

  • Theorem 1.1: Informal version of Theorem \ref{['thm:rmulmc-spacetime']}
  • Theorem 1.2: Informal version of Theorem \ref{['thm:rmulmc_cvx']}, $\alpha > 0$ case
  • Definition 2.1: Rényi divergence
  • Proposition 2.2: Data processing inequality
  • Proposition 2.3: Weak triangle inequality
  • Theorem 2.4: Shifted chain rule
  • Theorem 3.2: Harnack inequalities for the underdamped Langevin dynamics
  • Remark 3.3: Short-time vs. long-time asymptotics
  • Remark 3.4: Exponential decay in the strongly convex case
  • Remark 3.5: Random initializations
  • ...and 76 more