On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimates
Stefano Bruno, Ying Zhang, Dong-Young Lim, Ömer Deniz Akyildiz, Sotirios Sabanis
TL;DR
This paper provides non-asymptotic convergence guarantees for diffusion-based score-based generative models under strongly log-concave data. It introduces an auxiliary backward process that depends only on known information and analyzes score estimation via a Lipschitz, time-dependent approximator, linking optimization and sampling to yield explicit Wasserstein-2 bounds. In a motivating Gaussian-with-unknown-mean example, the authors obtain an optimal rate of order one with dimension dependence $\sqrt{d}$ and explicit constants, using SGLD for score optimization. For the general case, they derive bounds of the form $W_2\le C_1\sqrt{\varepsilon}+C_2 e^{-2\widehat{L}_{\text{MO}}(T-\varepsilon)-\varepsilon}+C_3(T,\varepsilon)\sqrt{\varepsilon_{\text{SN}}}+C_4(T,\varepsilon)\gamma^{\alpha}$, showing that appropriate choices of $(\varepsilon, T, \varepsilon_{\text{SN}}, \gamma)$ yield arbitrarily small error and highlighting improved dimension dependence under relaxed smoothness assumptions. The work thereby provides state-of-the-art, explicit convergence guarantees that quantify how diffusion sampling error scales with dimension, time discretization, and score-approximation quality, with practical implications for algorithm design and optimization in SGMs.
Abstract
We provide full theoretical guarantees for the convergence behaviour of diffusion-based generative models under the assumption of strongly log-concave data distributions while our approximating class of functions used for score estimation is made of Lipschitz continuous functions avoiding any Lipschitzness assumption on the score function. We demonstrate via a motivating example, sampling from a Gaussian distribution with unknown mean, the powerfulness of our approach. In this case, explicit estimates are provided for the associated optimization problem, i.e. score approximation, while these are combined with the corresponding sampling estimates. As a result, we obtain the best known upper bound estimates in terms of key quantities of interest, such as the dimension and rates of convergence, for the Wasserstein-2 distance between the data distribution (Gaussian with unknown mean) and our sampling algorithm. Beyond the motivating example and in order to allow for the use of a diverse range of stochastic optimizers, we present our results using an $L^2$-accurate score estimation assumption, which crucially is formed under an expectation with respect to the stochastic optimizer and our novel auxiliary process that uses only known information. This approach yields the best known convergence rate for our sampling algorithm.