Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Provable Acceleration for Diffusion Models under Minimal Assumptions

Gen Li, Changxiao Cai

TL;DR

This work tackles the slow sampling of score-based diffusion models by proposing a training-free accelerated sampler that operates under minimal assumptions: only $L^2$-accurate score estimates and finite second moments of the target distribution. The approach injects randomness and employs a momentum-like, second-order correction derived from higher-order approximations to the probability flow ODE, combined with clipping to control large errors. The authors prove that the sampler achieves $\varepsilon$-accuracy in total variation within $\widetilde{O}(d^{5/4}/\sqrt{\varepsilon})$ iterations (up to a burn-in cost), improving over the standard $\widetilde{O}(d/\varepsilon)$ rate for $\varepsilon \le 1/\sqrt{d}$, without requiring distributional or high-order smoothness assumptions. This represents the first provable acceleration for score-based samplers under such relaxed conditions, offering practical speedups for high-dimensional diffusion-model sampling.

Abstract

Score-based diffusion models, while achieving minimax optimality for sampling, are often hampered by slow sampling speeds due to the high computational burden of score function evaluations. Despite the recent remarkable empirical advances in speeding up the score-based samplers, theoretical understanding of acceleration techniques remains largely limited. To bridge this gap, we propose a novel training-free acceleration scheme for stochastic samplers. Under minimal assumptions -- namely, $L^2$-accurate score estimates and a finite second-moment condition on the target distribution -- our accelerated sampler provably achieves $\varepsilon$-accuracy in total variation within $\widetilde{O}(d^{5/4}/\sqrt{\varepsilon})$ iterations, thereby significantly improving upon the $\widetilde{O}(d/\varepsilon)$ iteration complexity of standard score-based samplers for $\varepsilon\leq 1/\sqrt{d}$. Notably, our convergence theory does not rely on restrictive assumptions on the target distribution or higher-order score estimation guarantees.

Provable Acceleration for Diffusion Models under Minimal Assumptions

TL;DR

This work tackles the slow sampling of score-based diffusion models by proposing a training-free accelerated sampler that operates under minimal assumptions: only -accurate score estimates and finite second moments of the target distribution. The approach injects randomness and employs a momentum-like, second-order correction derived from higher-order approximations to the probability flow ODE, combined with clipping to control large errors. The authors prove that the sampler achieves -accuracy in total variation within iterations (up to a burn-in cost), improving over the standard rate for , without requiring distributional or high-order smoothness assumptions. This represents the first provable acceleration for score-based samplers under such relaxed conditions, offering practical speedups for high-dimensional diffusion-model sampling.

Abstract

Score-based diffusion models, while achieving minimax optimality for sampling, are often hampered by slow sampling speeds due to the high computational burden of score function evaluations. Despite the recent remarkable empirical advances in speeding up the score-based samplers, theoretical understanding of acceleration techniques remains largely limited. To bridge this gap, we propose a novel training-free acceleration scheme for stochastic samplers. Under minimal assumptions -- namely, -accurate score estimates and a finite second-moment condition on the target distribution -- our accelerated sampler provably achieves -accuracy in total variation within iterations, thereby significantly improving upon the iteration complexity of standard score-based samplers for . Notably, our convergence theory does not rely on restrictive assumptions on the target distribution or higher-order score estimation guarantees.

Paper Structure

This paper contains 48 sections, 128 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Contributions
  3. Other related work
  4. Notation
  5. Organization
  6. Problem formulation
  7. Preliminaries
  8. Assumptions
  9. Main results
  10. Accelerated sampler
  11. Convergence guarantee
  12. Analysis
  13. Preliminaries
  14. Preparation.
  15. Step 1: constructing auxiliary processes
  16. ...and 33 more sections

Figures (1)

  • Figure 1: Sampling error of the proposed acceleration method, DDPM, and fitted rate $T\mapsto\Theta(\log^4 T /T^4)$: (a) $k=10,d=10$; (b) $k=10,d=100$; (c) $k=100,d=500$.

Theorems & Definitions (3)

  • proof
  • proof
  • proof