Convergence Analysis of Probability Flow ODE for Score-based Generative Models

TL;DR

This work analyzes the convergence of deterministic samplers based on probability flow ODEs for score-based generative models, focusing on how score estimation error and time discretization affect sampling fidelity. It establishes a continuous-time total-variation bound of O($d^{3/4}\delta^{1/2}$) under mild assumptions, and a discretized bound combining score error and RK discretization error as O($d^{3/4}\delta^{1/2} + d\cdot(dh)^p$). An iteration complexity of O($d^{1+1/p}\varepsilon^{-1/p}$) is derived for achieving a target TV accuracy, and the theory is validated through numerical experiments on Gaussian mixtures up to 128 dimensions. The results leverage transport along characteristics, interpolation-based discretization analysis, and dimension-free Gagliardo–Nirenberg inequalities, with extensions to general transport equations. Overall, the paper provides both theoretical guarantees and empirical evidence for the stability and efficiency of probability-flow-based score models under nonideal score estimates.

Abstract

Score-based generative models have emerged as a powerful approach for sampling high-dimensional probability distributions. Despite their effectiveness, their theoretical underpinnings remain relatively underdeveloped. In this work, we study the convergence properties of deterministic samplers based on probability flow ODEs from both theoretical and numerical perspectives. Assuming access to $L^2$-accurate estimates of the score function, we prove the total variation between the target and the generated data distributions can be bounded above by $\mathcal{O}(d^{3/4}δ^{1/2})$ in the continuous time level, where $d$ denotes the data dimension and $δ$ represents the $L^2$-score matching error. For practical implementations using a $p$-th order Runge-Kutta integrator with step size $h$, we establish error bounds of $\mathcal{O}(d^{3/4}δ^{1/2} + d\cdot(dh)^p)$ at the discrete level. Finally, we present numerical studies on problems up to 128 dimensions to verify our theory.

Figures (7)

• Figure 1: One dimensional test: Density estimations obtained by solving the Fokker-Planck PDE \ref{['e:defUV']} numerically with various artificial score errors. From top to bottom: score, estimated density with $\delta = 0.005,\,0.01,\,0.02$. From left to right estimated $q_t$ at $t = 8,\,4,\,2,\,1,\,0$.
• Figure 2: One dimensional test: convergence of the density estimations obtained by solving the Fokker-Planck PDE \ref{['e:defUV']} numerically with various artificial score errors.
• Figure 3: One dimensional test: density estimations obtained by solving the probability flow ODE with Heun's method with various artificial score errors. From top to bottom: score, estimated density with $\delta = 0.005,\,0.01,\,0.02$. From left to right estimated $q_t$ at $t = 8,\,4,\,2,\,1,\,0$.
• Figure 4: One dimensional test: convergence of the density estimations obtained by solving the probability flow ODE with Heun's method, where $h^2 \approx \delta$ with various artificial score errors.
• Figure 5: 128 dimensional test: marginal densities obtained by solving the probability flow ODE with Heun's method with various artificial score errors. From top to bottom: score, estimated densities with $\delta = 0.005,\,0.01,\,0.02$. From left to right estimated $q_t$ at $t = 8,\,4,\,2,\,1,\,0$.

Theorems & Definitions (38)

• Remark 2.1
• Remark 3.4
• Theorem 3.5
• Remark 3.6
• Remark 3.7
• Remark 3.9
• Theorem 3.10
• Remark 3.11
• Remark 3.12
• Theorem A.1