Adaptivity and Convergence of Probability Flow ODEs in Diffusion Generative Models
Jiaqi Tang, Yuling Yan
TL;DR
This work analyzes the probability flow ODE sampler within score-based diffusion models and proves that, with a carefully chosen coefficient $\eta_t^*$, the total variation distance between the data distribution and the sampler scales as $\mathsf{TV}(p_{X_1}, p_{Y_1}) \lesssim c\frac{(k+\log d)\log^3 T}{T} + c(\varepsilon_{\mathsf{score}}+\varepsilon_{\mathsf{Jacobi}})\log T$, where $k$ is the intrinsic dimension defined via metric entropy and $d$ is the ambient dimension. This demonstrates automatic adaptivity of the deterministic probability flow ODE to low-dimensional structures, achieving an effectively dimension-free rate $O(k/T)$ when score estimation errors are negligible. The authors develop a streamlined proof framework, relying on high-probability typical-sets, a careful density-ratio analysis along the reverse trajectory, and a tripartite TV-distance bound, and they discuss how the result compares to related work and potential extensions to Wasserstein distance. Practically, the findings justify using probability flow samplers in data with low-dimensional structure, potentially accelerating sampling in diffusion-based generative models.
Abstract
Score-based generative models, which transform noise into data by learning to reverse a diffusion process, have become a cornerstone of modern generative AI. This paper contributes to establishing theoretical guarantees for the probability flow ODE, a widely used diffusion-based sampler known for its practical efficiency. While a number of prior works address its general convergence theory, it remains unclear whether the probability flow ODE sampler can adapt to the low-dimensional structures commonly present in natural image data. We demonstrate that, with accurate score function estimation, the probability flow ODE sampler achieves a convergence rate of $O(k/T)$ in total variation distance (ignoring logarithmic factors), where $k$ is the intrinsic dimension of the target distribution and $T$ is the number of iterations. This dimension-free convergence rate improves upon existing results that scale with the typically much larger ambient dimension, highlighting the ability of the probability flow ODE sampler to exploit intrinsic low-dimensional structures in the target distribution for faster sampling.