Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distance

Eliot Beyler, Francis Bach

TL;DR

This work provides a unified, simple framework to bound Wasserstein-2 convergence for both stochastic (DDPM-like) and deterministic (DDIM-like) diffusion-model samplers. It delivers the first Wasserstein bound for the Heun sampler and refines Euler-based bounds for the probability-flow ODE by leveraging spatial regularity of the learned score and denoising-score-matching principles. A key insight is that controlling the score error with respect to the true reverse process, together with smoothed-Wasserstein initialization bounds, yields sharp, interpretable rates and highlights the benefits of deterministic samplers in low-discretization regimes. The results bridge theory and practice by clarifying how initialization, discretization, and score-estimation errors interact under bounded-support or Gaussian-perturbed data, offering guidance for reliable diffusion-model sampling. These contributions advance understanding of diffusion-model convergence in Wasserstein distance and suggest practical directions for tighter bounds and higher-order parametrizations.

Abstract

We provide new convergence guarantees in Wasserstein distance for diffusion-based generative models, covering both stochastic (DDPM-like) and deterministic (DDIM-like) sampling methods. We introduce a simple framework to analyze discretization, initialization, and score estimation errors. Notably, we derive the first Wasserstein convergence bound for the Heun sampler and improve existing results for the Euler sampler of the probability flow ODE. Our analysis emphasizes the importance of spatial regularity of the learned score function and argues for controlling the score error with respect to the true reverse process, in line with denoising score matching. We also incorporate recent results on smoothed Wasserstein distances to sharpen initialization error bounds.

Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distance

TL;DR

This work provides a unified, simple framework to bound Wasserstein-2 convergence for both stochastic (DDPM-like) and deterministic (DDIM-like) diffusion-model samplers. It delivers the first Wasserstein bound for the Heun sampler and refines Euler-based bounds for the probability-flow ODE by leveraging spatial regularity of the learned score and denoising-score-matching principles. A key insight is that controlling the score error with respect to the true reverse process, together with smoothed-Wasserstein initialization bounds, yields sharp, interpretable rates and highlights the benefits of deterministic samplers in low-discretization regimes. The results bridge theory and practice by clarifying how initialization, discretization, and score-estimation errors interact under bounded-support or Gaussian-perturbed data, offering guidance for reliable diffusion-model sampling. These contributions advance understanding of diffusion-model convergence in Wasserstein distance and suggest practical directions for tighter bounds and higher-order parametrizations.

Abstract

We provide new convergence guarantees in Wasserstein distance for diffusion-based generative models, covering both stochastic (DDPM-like) and deterministic (DDIM-like) sampling methods. We introduce a simple framework to analyze discretization, initialization, and score estimation errors. Notably, we derive the first Wasserstein convergence bound for the Heun sampler and improve existing results for the Euler sampler of the probability flow ODE. Our analysis emphasizes the importance of spatial regularity of the learned score function and argues for controlling the score error with respect to the true reverse process, in line with denoising score matching. We also incorporate recent results on smoothed Wasserstein distances to sharpen initialization error bounds.

Paper Structure

This paper contains 50 sections, 20 theorems, 299 equations, 2 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Convergence Guarantees for Diffusion Models
  3. Wasserstein Distance
  4. Contributions
  5. Notation
  6. Preliminaries: Algorithms
  7. Defining Diffusion Processes
  8. Euler Discretization and Sampling Algorithms
  9. Controlling the Different Sources of Error in Diffusion Models
  10. Control of the Spatial Regularity of the Score and Propagation of Errors
  11. Control of the Discretization Error
  12. Control of Initialization Error
  13. Control of Early Stopping Error
  14. Sketch of the Proof Strategy and Discussion on Score Error Assumptions
  15. Why are Lipchitz-continuity Assumptions Important?
  16. ...and 35 more sections

Key Result

Proposition 1

We define a process $\overleftarrow{X}\!_t$, for $t \in [0,T]$, with the following stochastic differential equation: where $\overleftarrow{p}\!_t = p_{T-t}$ and $W_t$ is a Brownian motion. Then, $\overleftarrow{X}\!_t$ has the same marginal distributions as $X_{T-t}$, i.e., $\forall t \in [0,T], \mathcal{L}(\overleftarrow{X}\!_t) = \mathcal{L}( X_{T-t})$.

Figures (2)

  • Figure 1: Illustration of the 3 different characteristic times in the diffusion process.
  • Figure 2: Illustration of two different ways to decompose error at each step.

Theorems & Definitions (20)

  • Proposition 1
  • Proposition 2
  • Lemma 3
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • Proposition 7
  • Corollary 8
  • Lemma 9
  • Proposition 10
  • ...and 10 more