Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Zeroth-order Logconcave Sampling

Yunbum Kook, Santosh S. Vempala

Abstract

We study the zeroth-order query complexity of sampling from a general logconcave distribution: given access to an evaluation oracle for a convex function $V:\mathbb{R}^{d}\rightarrow\mathbb{R}\cup\{\infty\}$, output a point from a distribution within $\varepsilon$-distance to the density proportional to $e^{-V}$. A long line of work provides efficient algorithms for this problem in TV distance, assuming a pointwise warm start (i.e., in $\infty$-Rényi divergence), and using annealing to generate such a warm start. Here, we address the natural and more general problem of using a $q$-Rényi divergence warm start to generate a sample that is $\varepsilon$-close in $q$-Rényi divergence. Our first main result is an algorithm with this end-to-end guarantee with state-of-the-art complexity for $q=\widetildeΩ(1)$. Our second result shows how to generate a $q$-Rényi divergence warm start directly via annealing, by maintaining $q$-Rényi divergence throughout, thereby obtaining a streamlined analysis and improved complexity. Such results were previously known only under the stronger assumptions of smoothness and access to first-order oracles. We also show a lower bound for Gaussian annealing by disproving a geometric conjecture about quadratic tilts of isotropic logconcave distributions. Central to our approach, we establish hypercontractivity of the heat adjoint and translate this into improved mixing time guarantees for the Proximal Sampler. The resulting analysis of both sampling and annealing follows a simplified and natural path, directly tying convergence rates to isoperimetric constants of the target distribution.

Zeroth-order Logconcave Sampling

Abstract

We study the zeroth-order query complexity of sampling from a general logconcave distribution: given access to an evaluation oracle for a convex function , output a point from a distribution within -distance to the density proportional to . A long line of work provides efficient algorithms for this problem in TV distance, assuming a pointwise warm start (i.e., in -Rényi divergence), and using annealing to generate such a warm start. Here, we address the natural and more general problem of using a -Rényi divergence warm start to generate a sample that is -close in -Rényi divergence. Our first main result is an algorithm with this end-to-end guarantee with state-of-the-art complexity for . Our second result shows how to generate a -Rényi divergence warm start directly via annealing, by maintaining -Rényi divergence throughout, thereby obtaining a streamlined analysis and improved complexity. Such results were previously known only under the stronger assumptions of smoothness and access to first-order oracles. We also show a lower bound for Gaussian annealing by disproving a geometric conjecture about quadratic tilts of isotropic logconcave distributions. Central to our approach, we establish hypercontractivity of the heat adjoint and translate this into improved mixing time guarantees for the Proximal Sampler. The resulting analysis of both sampling and annealing follows a simplified and natural path, directly tying convergence rates to isoperimetric constants of the target distribution.

Paper Structure

This paper contains 92 sections, 29 theorems, 199 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Background and results
  3. Uniform sampling from a warm start
  4. Classical algorithms: Ball walk and Hit-and-run.
  5. Sampling under smoothness.
  6. The proximal sampler.
  7. Proximal sampler for uniform sampling.
  8. Proximal sampler with restart for balanced guarantees (§\ref{['sec:ps-unif']}).
  9. Improved warm-start generation via annealing
  10. Warm-start generation in the well-conditioned setting.
  11. Simulated annealing.
  12. Towards genuine warmness.
  13. $\mathsf{TV}$-collapse.
  14. Boosting the Rényi divergence order.
  15. Hypercontractivity of semigroups.
  16. ...and 77 more sections

Key Result

Theorem 1.1

Let $\pi$ be the uniform distribution over a convex body $\mathcal{K}\subset\mathbb{R}^{d}$ given by a membership oracle with $B_{1}\subset\mathcal{K}$ and $\Lambda=\lVert\operatorname{cov}\pi\rVert$. Given $\varepsilon\in(0,1/100)$, an initial distribution $\pi_{0}$, and $q\geq2\vee\widetilde{\Omeg

Figures (1)

  • Figure 1.1: Zeroth-order query complexities for logconcave sampling. Arrows indicate the sampling complexity from an $O(1)$-warm start. Complexities listed next to each left node indicate warmness-generation costs. The dagger sign ($\dagger$) indicates results for uniform sampling over convex bodies (membership oracle). Results without $\dagger$ hold for general logconcave distributions (evaluation oracle). Here, $\bar{R}:=R\vee1$, $\bar{\lambda}:=\lambda\vee1$, $\lambda:=\lVert\operatorname{cov}\pi\rVert$, and $R^{2}:=\mathbb{E}_{\pi}[|\cdot|^{2}]$ (so $\lambda\leq R^{2}$). Note that $\mathsf{TV}$-warmness can be treated as $\EuScript{R}_{\infty}$-warmness at the cost of collapsing downstream guarantees to $\mathsf{TV}$-distance.

Theorems & Definitions (57)

  • Theorem 1.1: Uniform sampling from a warm start
  • Theorem 1.2: Hypercontractivity under heat flow
  • Corollary 1.3: Hypercontractivity of proximal sampler
  • Theorem 1.4
  • Theorem 1.5
  • Corollary 1.6
  • Theorem 1.7: Zeroth-order logconcave sampling from a warm start
  • Theorem 1.8
  • Corollary 1.9
  • Lemma 2.1
  • ...and 47 more