Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

In-and-Out: Algorithmic Diffusion for Sampling Convex Bodies

Yunbum Kook, Santosh S. Vempala, Matthew S. Zhang

TL;DR

This work introduces In-and-Out, a diffusion-based sampling scheme for uniformly drawing from a high-dimensional convex body $\mathcal{K} \subset \mathbb{R}^d$ that achieves end-to-end guarantees in the $q$-Rényi divergence $\mathcal{R}_q$ and attains state-of-the-art runtime. The method constructs a two-step Gibbs sampler on an augmented distribution $\pi(x,y) \propto \exp(-\lvert x-y\rvert^2/(2h)) \mathbf{1}_{\mathcal{K}}(x)$, interpreting the forward and backward steps as heat flows that naturally contract toward the target distribution, with rigorous contraction bounds derived via Poincaré and log-Sobolev inequalities. The main results show that, from an $M$-warm start, the algorithm achieves $\mathcal{R}_q(\mu^X \| \pi^{\mathcal{K}}) \le \varepsilon$ with probability at least $1-\eta$ using $\widetilde{O}(q d^2 \Lambda \log^2 M /(\eta \varepsilon))$ proper steps and $\widetilde{O}(qM d^2 \Lambda \log^6(1/(\eta \varepsilon)))$ membership queries, where $\Lambda$ is the largest eigenvalue of the target covariance; the parameters $h$ and the rejection threshold $N$ are chosen accordingly. The framework unifies diffusion-based and geometric approaches, offers stronger $\mathcal{R}_q$ guarantees (implying TV, $W_2$, KL, and $\chi^2$ guarantees), and provides practical warm-start and isotropic-rounding considerations. The results extend the literature on constrained logconcave sampling by delivering end-to-end diffusion-based guarantees with competitive complexity, and they open avenues for differential-privacy-friendly sampling via Rényi divergences.

Abstract

We present a new random walk for uniformly sampling high-dimensional convex bodies. It achieves state-of-the-art runtime complexity with stronger guarantees on the output than previously known, namely in Rényi divergence (which implies TV, $\mathcal{W}_2$, KL, $χ^2$). The proof departs from known approaches for polytime algorithms for the problem -- we utilize a stochastic diffusion perspective to show contraction to the target distribution with the rate of convergence determined by functional isoperimetric constants of the target distribution.

In-and-Out: Algorithmic Diffusion for Sampling Convex Bodies

TL;DR

This work introduces In-and-Out, a diffusion-based sampling scheme for uniformly drawing from a high-dimensional convex body that achieves end-to-end guarantees in the -Rényi divergence and attains state-of-the-art runtime. The method constructs a two-step Gibbs sampler on an augmented distribution , interpreting the forward and backward steps as heat flows that naturally contract toward the target distribution, with rigorous contraction bounds derived via Poincaré and log-Sobolev inequalities. The main results show that, from an -warm start, the algorithm achieves with probability at least using proper steps and membership queries, where is the largest eigenvalue of the target covariance; the parameters and the rejection threshold are chosen accordingly. The framework unifies diffusion-based and geometric approaches, offers stronger guarantees (implying TV, , KL, and guarantees), and provides practical warm-start and isotropic-rounding considerations. The results extend the literature on constrained logconcave sampling by delivering end-to-end diffusion-based guarantees with competitive complexity, and they open avenues for differential-privacy-friendly sampling via Rényi divergences.

Abstract

We present a new random walk for uniformly sampling high-dimensional convex bodies. It achieves state-of-the-art runtime complexity with stronger guarantees on the output than previously known, namely in Rényi divergence (which implies TV, , KL, ). The proof departs from known approaches for polytime algorithms for the problem -- we utilize a stochastic diffusion perspective to show contraction to the target distribution with the rate of convergence determined by functional isoperimetric constants of the target distribution.
Paper Structure (21 sections, 27 theorems, 82 equations, 1 figure, 1 table, 3 algorithms)

This paper contains 21 sections, 27 theorems, 82 equations, 1 figure, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Diffusion for uniform sampling
  3. Results
  4. Discussion
  5. Related work
  6. Preliminaries
  7. Analysis
  8. Functional inequalities
  9. Contraction and mixing
  10. Failure probability and wasted steps
  11. Putting it together
  12. Acknowledgements.
  13. Ball walk and Speedy walk
  14. Direct analysis.
  15. Indirect analysis through $\mathsf{Speedy\ walk}$.
  16. ...and 6 more sections

Key Result

Theorem 3

For any given $\eta, \varepsilon \in (0,1)$, $q\geq 1$, and any convex body $\mathcal{K}$ given by a well-defined membership oracle, there exist choices of parameters $h, N$ such that $\mathsf{In\text{-}and\text{-}Out}$, starting from an $M$-warm distribution, with probability at least $1-\eta$, ret

Figures (1)

  • Figure 1.1: Description of uniform samplers: (i) $\mathsf{Ball\ walk}$: proposes a uniform random point $z$ from $B_{\delta}(x_1)$, but $z\notin \mathcal{K}$ so it stays at $x_1=x_2$. (ii) $\mathsf{Speedy\ walk}$: moves to $x_2$ drawn uniformly at random from $\mathcal{K} \cap B_{\delta}(x_1)$. (iii) $\mathsf{In\text{-}and\text{-}Out}$: first moves to $y_2$ obtained by taking a Gaussian step from $x_1$, and then to $x_2$ obtained by sampling the truncated Gaussian $\mathcal{N}(y_2,hI_d)|_{\mathcal{K}}$.

Theorems & Definitions (46)

  • Remark 1
  • Definition 1: Convex body oracle
  • Definition 2: Warmness
  • Theorem 3: Succinct version of Theorem \ref{['thm:main-result']}
  • Remark 2: Warm-start generation
  • Corollary 4
  • Theorem 5
  • Definition 6: Distance and divergence
  • Definition 7
  • Definition 8
  • ...and 36 more