Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Modeling Score Approximation Errors in Diffusion Models via Forward SPDEs

Junsu Seo

TL;DR

The paper addresses robustness in score-based diffusion models by modeling score estimation error as a stochastic source within a forward SPDE that governs the evolution of the density $u_t$ through the Fokker-Planck equation. It connects this SPDE formulation to geometric stability via displacement convexity in Wasserstein space and introduces SIEM, a metric derived from the SPDE’s quadratic variation projected onto a radial test function, which can be estimated efficiently. The study provides a simplified analysis showing how gradient-flow (KL) dissipation can dominate noise under convexity conditions, and validates SIEM against standard metrics (FID and $W_2$) on CIFAR-10 and LSUN-Bedrooms (32$\times$32), including scenarios using only a subset of the diffusion timesteps. The results suggest SIEM is a practical, computation-friendly surrogate for tracking sample quality and model progress, offering a geometric perspective on why SGMs remain robust even with imperfect score estimates.

Abstract

This study investigates the dynamics of Score-based Generative Models (SGMs) by treating the score estimation error as a stochastic source driving the Fokker-Planck equation. Departing from particle-centric SDE analyses, we employ an SPDE framework to model the evolution of the probability density field under stochastic drift perturbations. Under a simplified setting, we utilize this framework to interpret the robustness of generative models through the lens of geometric stability and displacement convexity. Furthermore, we introduce a candidate evaluation metric derived from the quadratic variation of the SPDE solution projected onto a radial test function. Preliminary observations suggest that this metric remains effective using only the initial 10% of the sampling trajectory, indicating a potential for computational efficiency.

Modeling Score Approximation Errors in Diffusion Models via Forward SPDEs

TL;DR

The paper addresses robustness in score-based diffusion models by modeling score estimation error as a stochastic source within a forward SPDE that governs the evolution of the density through the Fokker-Planck equation. It connects this SPDE formulation to geometric stability via displacement convexity in Wasserstein space and introduces SIEM, a metric derived from the SPDE’s quadratic variation projected onto a radial test function, which can be estimated efficiently. The study provides a simplified analysis showing how gradient-flow (KL) dissipation can dominate noise under convexity conditions, and validates SIEM against standard metrics (FID and ) on CIFAR-10 and LSUN-Bedrooms (3232), including scenarios using only a subset of the diffusion timesteps. The results suggest SIEM is a practical, computation-friendly surrogate for tracking sample quality and model progress, offering a geometric perspective on why SGMs remain robust even with imperfect score estimates.

Abstract

This study investigates the dynamics of Score-based Generative Models (SGMs) by treating the score estimation error as a stochastic source driving the Fokker-Planck equation. Departing from particle-centric SDE analyses, we employ an SPDE framework to model the evolution of the probability density field under stochastic drift perturbations. Under a simplified setting, we utilize this framework to interpret the robustness of generative models through the lens of geometric stability and displacement convexity. Furthermore, we introduce a candidate evaluation metric derived from the quadratic variation of the SPDE solution projected onto a radial test function. Preliminary observations suggest that this metric remains effective using only the initial 10% of the sampling trajectory, indicating a potential for computational efficiency.
Paper Structure (38 sections, 25 equations, 2 figures, 2 tables)

This paper contains 38 sections, 25 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Notation
  3. Background
  4. Score-based Generative Models and the Fokker-Planck Equation
  5. Forward and Reverse SDEs.
  6. The Fokker-Planck Equation Perspective.
  7. Stochastic Analysis in Infinite Dimensions
  8. Conservative Stochastic Dynamics.
  9. Choice of Integration Scheme.
  10. Motivation and Formulation of the Forward SPDE
  11. Analysis of a Simplified Model for Robustness
  12. Geometric Stability via Displacement Convexity
  13. SPDE-Induced Evaluation Metric
  14. Experiments
  15. Experimental Setup
  16. ...and 23 more sections

Figures (2)

  • Figure 1: Visualization of robustness in the simplified model. The gray funnel and blue vector field illustrate the geometric contractivity, which acts as a restoring force towards the ideal path. Although the trajectory $u_t$ (orange) is continuously subjected to stochastic perturbations ("shaking"), the geometric stability confines the error, ensuring convergence to the target $v_T$.
  • Figure 2: Evolution of $\boldsymbol{\mu}_t$ (light green), SIEM (orange), FID (blue), and 2-Wasserstein distance (light purple) during U-Net training on CIFAR-10. Generated samples at various training steps are shown at the bottom.