Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial

Wenpin Tang, Hanyang Zhao

TL;DR

This technical tutorial unifies score-based diffusion models under a continuous-time SDE framework, detailing how time reversal yields backward dynamics and how score matching learns the crucial score function to enable practical sampling. It covers a spectrum of score matching techniques (explicit, implicit, sliced, and denoising), introduces probability-flow ODE samplers and consistency models for one-shot generation, and surveys convergence analyses in total variation and Wasserstein distances along with discretization considerations. The paper also discusses statistical efficiency, neural-network implementations of score functions, and reinforcement-learning-inspired approaches to diffusion alignment, highlighting theoretical results and practical trade-offs. Collectively, it clarifies the theoretical backbone of diffusion models and outlines gaps and directions for robust, scalable, and task-aware diffusion-based generation.

Abstract

This is an expository article on the score-based diffusion models, with a particular focus on the formulation via stochastic differential equations (SDE). After a gentle introduction, we discuss the two pillars in the diffusion modeling -- sampling and score matching, which encompass the SDE/ODE sampling, score matching efficiency, the consistency models, and reinforcement learning. Short proofs are given to illustrate the main idea of the stated results. The article is primarily a technical introduction to the field, and practitioners may also find some analysis useful in designing new models or algorithms.

Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial

TL;DR

This technical tutorial unifies score-based diffusion models under a continuous-time SDE framework, detailing how time reversal yields backward dynamics and how score matching learns the crucial score function to enable practical sampling. It covers a spectrum of score matching techniques (explicit, implicit, sliced, and denoising), introduces probability-flow ODE samplers and consistency models for one-shot generation, and surveys convergence analyses in total variation and Wasserstein distances along with discretization considerations. The paper also discusses statistical efficiency, neural-network implementations of score functions, and reinforcement-learning-inspired approaches to diffusion alignment, highlighting theoretical results and practical trade-offs. Collectively, it clarifies the theoretical backbone of diffusion models and outlines gaps and directions for robust, scalable, and task-aware diffusion-based generation.

Abstract

This is an expository article on the score-based diffusion models, with a particular focus on the formulation via stochastic differential equations (SDE). After a gentle introduction, we discuss the two pillars in the diffusion modeling -- sampling and score matching, which encompass the SDE/ODE sampling, score matching efficiency, the consistency models, and reinforcement learning. Short proofs are given to illustrate the main idea of the stated results. The article is primarily a technical introduction to the field, and practitioners may also find some analysis useful in designing new models or algorithms.
Paper Structure (19 sections, 12 theorems, 129 equations, 1 figure)

This paper contains 19 sections, 12 theorems, 129 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Time reversal formula
  3. Examples
  4. Score matching techniques
  5. Score matching
  6. Sliced score matching
  7. Denoising score matching
  8. ODE samplers and consistency model
  9. The probability flow ODE
  10. Consistency model
  11. Convergence results
  12. Stochastic sampler -- total variation bound
  13. Stochastic sampler -- Wasserstein bound
  14. Discretization
  15. Further results on score matching
  16. ...and 4 more sections

Key Result

Theorem 2.1

Ander82HP86 Under suitable conditions on $f(\cdot, \cdot)$, $g(\cdot, \cdot)$ and $\{p(t, \cdot)\}_{0 \le t \le T}$, we have:

Figures (1)

  • Figure 1: Swiss Roll generation.

Theorems & Definitions (22)

  • Theorem 2.1: Time reversal formula
  • proof
  • Theorem 4.1: Implicit score matching
  • proof
  • Theorem 4.2: Sliced score matching
  • Theorem 4.3: Denoising score matching
  • proof
  • Theorem 5.1: Probability flow ODE
  • proof
  • Theorem 6.2: Total variation bound
  • ...and 12 more