Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding Diffusion Models by Feynman's Path Integral

Yuji Hirono, Akinori Tanaka, Kenji Fukushima

TL;DR

The paper presents a path-integral formulation of diffusion models based on Feynman’s path integral, linking stochastic score-based diffusion and deterministic probability-flow ODEs through an interpolating parameter $\mathfrak{h}$ that plays the role of Planck’s constant. It re-derives backward-time dynamics, loss functions via Girsanov, and provides a KL-based training objective within this framework. A Wentzel–Kramers–Brillouin (WKB) expansion in $\mathfrak{h}$ yields first-order corrections to the log-likelihood, enabling quantitative analysis of how noise influences sampling performance and model likelihood. Empirical studies on 1D and 2D toy datasets validate the theory and illustrate how controlled noise can improve likelihood under imperfect score estimation, while offering a principled method to compare stochastic and deterministic sampling via $W_2$ distances. The framework opens avenues for analyzing diffusion Schrödinger bridges and priors beyond standard Gaussian assumptions, with implications for understanding when and why noise helps diffusion-based generation.

Abstract

Score-based diffusion models have proven effective in image generation and have gained widespread usage; however, the underlying factors contributing to the performance disparity between stochastic and deterministic (i.e., the probability flow ODEs) sampling schemes remain unclear. We introduce a novel formulation of diffusion models using Feynman's path integral, which is a formulation originally developed for quantum physics. We find this formulation providing comprehensive descriptions of score-based generative models, and demonstrate the derivation of backward stochastic differential equations and loss functions.The formulation accommodates an interpolating parameter connecting stochastic and deterministic sampling schemes, and we identify this parameter as a counterpart of Planck's constant in quantum physics. This analogy enables us to apply the Wentzel-Kramers-Brillouin (WKB) expansion, a well-established technique in quantum physics, for evaluating the negative log-likelihood to assess the performance disparity between stochastic and deterministic sampling schemes.

Understanding Diffusion Models by Feynman's Path Integral

TL;DR

The paper presents a path-integral formulation of diffusion models based on Feynman’s path integral, linking stochastic score-based diffusion and deterministic probability-flow ODEs through an interpolating parameter that plays the role of Planck’s constant. It re-derives backward-time dynamics, loss functions via Girsanov, and provides a KL-based training objective within this framework. A Wentzel–Kramers–Brillouin (WKB) expansion in yields first-order corrections to the log-likelihood, enabling quantitative analysis of how noise influences sampling performance and model likelihood. Empirical studies on 1D and 2D toy datasets validate the theory and illustrate how controlled noise can improve likelihood under imperfect score estimation, while offering a principled method to compare stochastic and deterministic sampling via distances. The framework opens avenues for analyzing diffusion Schrödinger bridges and priors beyond standard Gaussian assumptions, with implications for understanding when and why noise helps diffusion-based generation.

Abstract

Score-based diffusion models have proven effective in image generation and have gained widespread usage; however, the underlying factors contributing to the performance disparity between stochastic and deterministic (i.e., the probability flow ODEs) sampling schemes remain unclear. We introduce a novel formulation of diffusion models using Feynman's path integral, which is a formulation originally developed for quantum physics. We find this formulation providing comprehensive descriptions of score-based generative models, and demonstrate the derivation of backward stochastic differential equations and loss functions.The formulation accommodates an interpolating parameter connecting stochastic and deterministic sampling schemes, and we identify this parameter as a counterpart of Planck's constant in quantum physics. This analogy enables us to apply the Wentzel-Kramers-Brillouin (WKB) expansion, a well-established technique in quantum physics, for evaluating the negative log-likelihood to assess the performance disparity between stochastic and deterministic sampling schemes.
Paper Structure (41 sections, 7 theorems, 142 equations, 11 figures, 1 table, 1 algorithm)

This paper contains 41 sections, 7 theorems, 142 equations, 11 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Diffusion models
  4. Likelihood calculation in diffusion models
  5. Stochastic and deterministic sampling procedures
  6. Reformulation of diffusion models by path integral formalism
  7. Forward and reverse processes
  8. Models and objectives
  9. Log-likelihood by WKB expansion
  10. Interpolating SDE and probability flow ODE
  11. $\mathfrak{h} \neq 0$ in path integral
  12. Experiments
  13. Analysis of 1-dim. Gaussian data
  14. Experiment of 2-dim. synthetic data
  15. Pretrained models
  16. ...and 26 more sections

Key Result

Proposition 3.1

The path-probability $P(\{\mathbf{x}_t\}_{t \in [0, T]})$ can be represented in the following path integral form: with $\mathcal{A} \coloneqq \int_0^T L(\dot{\mathbf{x}}_{t}, \mathbf{x}_{t}) {\rm d} t + J$, where $L(\dot{\mathbf{x}}_t, \mathbf{x}_t)$ is called Onsager–Machlup function PhysRev.91.1505 defined by and $J$ is the Jacobian associated with the chosen discretization scheme in stochasti

Figures (11)

  • Figure 1: Schematic of the path integral formulation of diffusion models.
  • Figure 2: Gray dots: training data. Colored lines: generative trajectories based on \ref{['eq:1para_gen_sde']} for different noise levels $\mathfrak{h}$ from the identical initial vector $\mathbf{x}_T$ shown by $\times$.
  • Figure 3: (Top) Negative log-likelihood (NLL) of the 1-dim. Gaussian toy model. (Bottom) 2-Wasserstein metric ($W_2$) between the data distribution and the distribution obtained by the same model. Both panels are plotted as a function of parameter $\mathfrak{h}$.
  • Figure 4: 2-Wasserstein metrics ($W_2$) by POT library flamary2021pot between validation data and generated data via \ref{['eq:1para_gen_sde']} with Swiss-roll data and 25-Gaussian data with SIMPLE and COSINE SDE scheduling. The dots and the errorbars represent the mean values over 10 independent trials and $\pm$std$/\sqrt{10}$, respectively.
  • Figure 5: (Left) 25-Gaussian data (3,000 samples), and (Right) Swiss-roll data (3,000 samples).
  • ...and 6 more figures

Theorems & Definitions (7)

  • Proposition 3.1
  • Proposition 3.2
  • Proposition 3.3
  • Theorem 4.1
  • Theorem 4.2
  • Proposition 1.1
  • Proposition 1.2