Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixtures
Gen Li, Changxiao Cai, Yuting Wei
TL;DR
This work analyzes diffusion models, particularly DDPM, for sampling distributions that can be well-approximated by isotropic Gaussian mixtures. It proves a non-asymptotic, dimension-free bound: with perfect score estimates and small GMM approximation error, DDPM achieves an $\varepsilon$-accurate distribution in total variation after at most $\tilde{O}(1/\varepsilon)$ iterations, independent of the ambient dimension $d$ and the number of components $K$ up to logarithmic factors. The result is robust to score estimation errors and decomposes the error into discretization, score estimation, and GMM approximation terms, providing a theoretical explanation for the empirically observed efficiency of diffusion models in high dimensions. This work suggests diffusion models exploit intrinsic structure (GMM-like decomposability) to achieve fast convergence in sampling, with potential implications for scalable generative modeling in ultra-high dimensions.
Abstract
Diffusion models are distinguished by their exceptional generative performance, particularly in producing high-quality samples through iterative denoising. While current theory suggests that the number of denoising steps required for accurate sample generation should scale linearly with data dimension, this does not reflect the practical efficiency of widely used algorithms like Denoising Diffusion Probabilistic Models (DDPMs). This paper investigates the effectiveness of diffusion models in sampling from complex high-dimensional distributions that can be well-approximated by Gaussian Mixture Models (GMMs). For these distributions, our main result shows that DDPM takes at most $\widetilde{O}(1/\varepsilon)$ iterations to attain an $\varepsilon$-accurate distribution in total variation (TV) distance, independent of both the ambient dimension $d$ and the number of components $K$, up to logarithmic factors. Furthermore, this result remains robust to score estimation errors. These findings highlight the remarkable effectiveness of diffusion models in high-dimensional settings given the universal approximation capability of GMMs, and provide theoretical insights into their practical success.