Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-accuracy log-concave sampling with stochastic queries

Fan Chen, Sinho Chewi, Constantinos Daskalakis, Alexander Rakhlin

TL;DR

This work shows that high-accuracy guarantees for log-concave sampling and query complexities which scale as $\mathrm{poly}(1/\delta)$ are achievable using stochastic gradients with subexponential tails, and provides similar high accuracy guarantees under stochastic zeroth order (value) queries.

Abstract

We show that high-accuracy guarantees for log-concave sampling -- that is, iteration and query complexities which scale as $\mathrm{poly}\log(1/δ)$, where $δ$ is the desired target accuracy -- are achievable using stochastic gradients with subexponential tails. Notably, this exhibits a separation with the problem of convex optimization, where stochasticity (even additive Gaussian noise) in the gradient oracle incurs $\mathrm{poly}(1/δ)$ queries. We also give an information-theoretic argument that light-tailed stochastic gradients are necessary for high accuracy: for example, in the bounded variance case, we show that the minimax-optimal query complexity scales as $Θ(1/δ)$. Our framework also provides similar high accuracy guarantees under stochastic zeroth order (value) queries.

High-accuracy log-concave sampling with stochastic queries

TL;DR

This work shows that high-accuracy guarantees for log-concave sampling and query complexities which scale as are achievable using stochastic gradients with subexponential tails, and provides similar high accuracy guarantees under stochastic zeroth order (value) queries.

Abstract

We show that high-accuracy guarantees for log-concave sampling -- that is, iteration and query complexities which scale as , where is the desired target accuracy -- are achievable using stochastic gradients with subexponential tails. Notably, this exhibits a separation with the problem of convex optimization, where stochasticity (even additive Gaussian noise) in the gradient oracle incurs queries. We also give an information-theoretic argument that light-tailed stochastic gradients are necessary for high accuracy: for example, in the bounded variance case, we show that the minimax-optimal query complexity scales as . Our framework also provides similar high accuracy guarantees under stochastic zeroth order (value) queries.
Paper Structure (21 sections, 15 theorems, 95 equations, 1 algorithm)

This paper contains 21 sections, 15 theorems, 95 equations, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Preliminaries
  4. Notation
  5. High-accuracy sampling with stochastic queries
  6. Background on first-order rejection sampling (FORS)
  7. Sampling from Gaussian tilts with stochastic queries
  8. Log-concave sampling
  9. Lower bound: light tails are necessary for high accuracy sampling
  10. A simple lower bound
  11. Revisiting the lower bound of ChaBarLon22Lower
  12. Conclusion
  13. Implementation of the approximate proximal oracle
  14. Technical tools
  15. Proofs from \ref{['sec:upper']}
  16. ...and 6 more sections

Key Result

Lemma 2.1

Suppose that ass:holder holds with $s\in[0,1]$ and denote $m_s=\beta_s^{1/(1+s)}$. Suppose that $\eta\leq \frac{1}{2m_s}$ and we are given access to a stochastic gradient oracle with $\epsilon$-tail, and that $\|\nabla f(x_0)\|\leq G$. Then, the approximate proximal oracle with $\varepsilon_{\mathsf

Theorems & Definitions (26)

  • Definition 1: Oracle with $\epsilon$-tail
  • Example 1: Sub-polynomial tail
  • Example 2: Polynomial tail
  • Lemma 2.1
  • Theorem 3.1: FORS guarantee, chen2026high
  • Remark 1: Diffusion models
  • Theorem 3.2: Sampling from Gaussian tilts
  • Remark 2: Dimension dependence
  • Remark 3: Proximal tolerance
  • Remark 4: Accuracy dependence
  • ...and 16 more