Closed-Form Diffusion Models
Christopher Scarvelis, Haitz Sáez de Ocáriz Borde, Justin Solomon
TL;DR
This work tackles the memorization and training-cost issues of score-based diffusion models by introducing smoothed closed-form diffusion models (σ-CFDMs) that generate novel samples without training. By explicitly smoothing the exact closed-form score, the method biases the sampling dynamics toward barycenters of nearby training points, enabling training-free generation with theoretical support on the sampling distribution’s support. An efficient NN-based estimator accelerates score evaluation, and the approach scales to high-dimensional data, achieving CPU-based image generation competitive with GPU-backed neural diffusion baselines, including notable results in pixel- and latent-space experiments. The findings highlight smoothing as a principled inductive bias for generalization and open avenues for conditioning and guidance without neural score learning.
Abstract
Score-based generative models (SGMs) sample from a target distribution by iteratively transforming noise using the score function of the perturbed target. For any finite training set, this score function can be evaluated in closed form, but the resulting SGM memorizes its training data and does not generate novel samples. In practice, one approximates the score by training a neural network via score-matching. The error in this approximation promotes generalization, but neural SGMs are costly to train and sample, and the effective regularization this error provides is not well-understood theoretically. In this work, we instead explicitly smooth the closed-form score to obtain an SGM that generates novel samples without training. We analyze our model and propose an efficient nearest-neighbor-based estimator of its score function. Using this estimator, our method achieves competitive sampling times while running on consumer-grade CPUs.