Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast parallel sampling under isoperimetry

Nima Anari, Sinho Chewi, Thuy-Duong Vuong

TL;DR

The paper develops fast parallel samplers for continuous distributions with log-Sobolev inequalities and smooth potentials by parallelizing LMC and ULMC through Picard iterations. It provides KL- and TV-distance guarantees with polylogarithmic parallel depth and near-linear gradient-query complexity, including a TV-guaranteed ULMC variant. The analysis leverages the interpolation method and Girsanov’s theorem, under LSI and smoothness assumptions, to bound discretization errors and achieve sharp distance guarantees. The work extends to discrete-distribution sampling via TV guarantees, enabling RNC sampling-to-counting reductions for hypercube distributions with bounded tilts and covariance, and yielding practical Rd computations for directed Eulerian tours and asymmetric determinantal point processes.

Abstract

We show how to sample in parallel from a distribution $π$ over $\mathbb R^d$ that satisfies a log-Sobolev inequality and has a smooth log-density, by parallelizing the Langevin (resp. underdamped Langevin) algorithms. We show that our algorithm outputs samples from a distribution $\hatπ$ that is close to $π$ in Kullback--Leibler (KL) divergence (resp. total variation (TV) distance), while using only $\log(d)^{O(1)}$ parallel rounds and $\widetilde{O}(d)$ (resp. $\widetilde O(\sqrt d)$) gradient evaluations in total. This constitutes the first parallel sampling algorithms with TV distance guarantees. For our main application, we show how to combine the TV distance guarantees of our algorithms with prior works and obtain RNC sampling-to-counting reductions for families of discrete distribution on the hypercube $\{\pm 1\}^n$ that are closed under exponential tilts and have bounded covariance. Consequently, we obtain an RNC sampler for directed Eulerian tours and asymmetric determinantal point processes, resolving open questions raised in prior works.

Fast parallel sampling under isoperimetry

TL;DR

The paper develops fast parallel samplers for continuous distributions with log-Sobolev inequalities and smooth potentials by parallelizing LMC and ULMC through Picard iterations. It provides KL- and TV-distance guarantees with polylogarithmic parallel depth and near-linear gradient-query complexity, including a TV-guaranteed ULMC variant. The analysis leverages the interpolation method and Girsanov’s theorem, under LSI and smoothness assumptions, to bound discretization errors and achieve sharp distance guarantees. The work extends to discrete-distribution sampling via TV guarantees, enabling RNC sampling-to-counting reductions for hypercube distributions with bounded tilts and covariance, and yielding practical Rd computations for directed Eulerian tours and asymmetric determinantal point processes.

Abstract

We show how to sample in parallel from a distribution over that satisfies a log-Sobolev inequality and has a smooth log-density, by parallelizing the Langevin (resp. underdamped Langevin) algorithms. We show that our algorithm outputs samples from a distribution that is close to in Kullback--Leibler (KL) divergence (resp. total variation (TV) distance), while using only parallel rounds and (resp. ) gradient evaluations in total. This constitutes the first parallel sampling algorithms with TV distance guarantees. For our main application, we show how to combine the TV distance guarantees of our algorithms with prior works and obtain RNC sampling-to-counting reductions for families of discrete distribution on the hypercube that are closed under exponential tilts and have bounded covariance. Consequently, we obtain an RNC sampler for directed Eulerian tours and asymmetric determinantal point processes, resolving open questions raised in prior works.
Paper Structure (16 sections, 14 theorems, 97 equations, 3 algorithms)

This paper contains 16 sections, 14 theorems, 97 equations, 3 algorithms.

Table of Contents

  1. Introduction
  2. Algorithm
  3. Analysis techniques
  4. Applications
  5. Preliminaries
  6. Log-concave distributions
  7. Log-Sobolev and transport-entropy inequalities
  8. Parallel sampling guarantees
  9. LMC
  10. ULMC
  11. Algorithm
  12. Analysis
  13. Implications for sampling from discrete distributions
  14. Proofs for LMC
  15. Proofs for ULMC
  16. ...and 1 more sections

Key Result

theorem 1

Suppose that $\pi=\exp(-V)$ is a density on $\R^d$ that satisfies a log-Sobolev inequality and has a smooth potential $V$. Assume that we are given (approximate) oracle access to $\nabla V$. Then, we can produce samples from a distribution $\hat{\pi}$ with the following guarantees.

Theorems & Definitions (31)

  • theorem 1: Informal main theorem
  • remark 1
  • definition 1: Approximate oracle for the Laplace transform
  • theorem 2
  • corollary 1
  • definition 2: Kullback--Leibler divergence
  • definition 3: Smoothness
  • definition 4: Strong log-concavity
  • definition 5: Log-Sobolev inequality
  • definition 6: Relative Fisher information
  • ...and 21 more