Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-accuracy sampling for diffusion models and log-concave distributions

Fan Chen, Sinho Chewi, Constantinos Daskalakis, Alexander Rakhlin

TL;DR

This work introduces FORS, a first-order rejection-sampling meta-algorithm, to achieve high-accuracy sampling using only gradient (score) evaluations. By instantiating FORS within diffusion models and in the log-concave setting, the authors derive polylogarithmic-in-$1/\delta$ sampling guarantees under minimal data assumptions, with improved dimension dependence under Lipschitz-score or intrinsic-dimension conditions. The framework yields diffusion samplers with δ-error in polylogarithmic steps and provides a gradient-only polylogarithmic sampler for general log-concave distributions via the proximal-sampler/RGO approach. These results significantly accelerate high-accuracy sampling without density evaluations, unifying diffusion-based and log-concave methods under a first-order information regime.

Abstract

We present algorithms for diffusion model sampling which obtain $δ$-error in $\mathrm{polylog}(1/δ)$ steps, given access to $\widetilde O(δ)$-accurate score estimates in $L^2$. This is an exponential improvement over all previous results. Specifically, under minimal data assumptions, the complexity is $\widetilde O(d\,\mathrm{polylog}(1/δ))$ where $d$ is the dimension of the data; under a non-uniform $L$-Lipschitz condition, the complexity is $\widetilde O(\sqrt{dL}\,\mathrm{polylog}(1/δ))$; and if the data distribution has intrinsic dimension $d_\star$, then the complexity reduces to $\widetilde O(d_\star\,\mathrm{polylog}(1/δ))$. Our approach also yields the first $\mathrm{polylog}(1/δ)$ complexity sampler for general log-concave distributions using only gradient evaluations.

High-accuracy sampling for diffusion models and log-concave distributions

TL;DR

This work introduces FORS, a first-order rejection-sampling meta-algorithm, to achieve high-accuracy sampling using only gradient (score) evaluations. By instantiating FORS within diffusion models and in the log-concave setting, the authors derive polylogarithmic-in- sampling guarantees under minimal data assumptions, with improved dimension dependence under Lipschitz-score or intrinsic-dimension conditions. The framework yields diffusion samplers with δ-error in polylogarithmic steps and provides a gradient-only polylogarithmic sampler for general log-concave distributions via the proximal-sampler/RGO approach. These results significantly accelerate high-accuracy sampling without density evaluations, unifying diffusion-based and log-concave methods under a first-order information regime.

Abstract

We present algorithms for diffusion model sampling which obtain -error in steps, given access to -accurate score estimates in . This is an exponential improvement over all previous results. Specifically, under minimal data assumptions, the complexity is where is the dimension of the data; under a non-uniform -Lipschitz condition, the complexity is ; and if the data distribution has intrinsic dimension , then the complexity reduces to . Our approach also yields the first complexity sampler for general log-concave distributions using only gradient evaluations.
Paper Structure (50 sections, 29 theorems, 222 equations, 3 algorithms)

This paper contains 50 sections, 29 theorems, 222 equations, 3 algorithms.

Table of Contents

  1. Introduction
  2. Our contribution
  3. Diffusion sampling
  4. Log-concave sampling
  5. Related work
  6. High-accuracy diffusion sampling
  7. Concurrent work
  8. Log-concave sampling
  9. Notation
  10. Background on diffusion models
  11. Score function and estimates
  12. Roadmap
  13. Key subroutine: Gaussian tilts
  14. First-order rejection sampling (FORS)
  15. Sampling from Gaussian tilts with an exact oracle
  16. ...and 35 more sections

Key Result

Theorem 3.1

alg:fors outputs a random point with density $\widehat{p}(x)\propto q(x)\, e^{\mathop{\mathrm{\mathbb{E}}}\nolimits[W_1\mid x]}$. The number of sampled $W_j$'s is bounded, with probability at least $1-\delta$, by $3Be^{2B}\log(2/\delta)$. Moreover, if alg:fors is called $T$ times, then with probabil

Theorems & Definitions (33)

  • Definition 2.1: Score estimation error
  • Theorem 3.1: FORS guarantee
  • Theorem 3.3
  • Theorem 4.1
  • Corollary 4.2
  • Theorem 4.4
  • Definition 4.5: Intrinsic dimension
  • Theorem 4.6
  • Definition B.1: PI
  • Definition B.2: LSI
  • ...and 23 more