Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions
Hongrui Chen, Holden Lee, Jianfeng Lu
TL;DR
This work advances the theoretical understanding of score-based diffusion models by showing that an $L^2$-accurate score estimator yields efficient convergence guarantees under minimal smoothness, without relying on log-concavity. It develops three regimes—trajectory-smooth, general non-smooth with early stopping, and smooth $p_0$—and analyzes discretization schemes to provide near-optimal step counts with either KL or Wasserstein-type guarantees. A key contribution is leveraging forward-process smoothing to obtain high-probability Hessian bounds, enabling discretization control and improved dependence (logarithmic in moments or smoothness) on problem parameters. The results offer practical discretization guidance and broaden the scope of SGMs to multi-modal or weakly smooth distributions, beyond prior polynomial-time bounds under stronger assumptions.
Abstract
We give an improved theoretical analysis of score-based generative modeling. Under a score estimate with small $L^2$ error (averaged across timesteps), we provide efficient convergence guarantees for any data distribution with second-order moment, by either employing early stopping or assuming smoothness condition on the score function of the data distribution. Our result does not rely on any log-concavity or functional inequality assumption and has a logarithmic dependence on the smoothness. In particular, we show that under only a finite second moment condition, approximating the following in reverse KL divergence in $ε$-accuracy can be done in $\tilde O\left(\frac{d \log (1/δ)}ε\right)$ steps: 1) the variance-$δ$ Gaussian perturbation of any data distribution; 2) data distributions with $1/δ$-smooth score functions. Our analysis also provides a quantitative comparison between different discrete approximations and may guide the choice of discretization points in practice.