A Practical Diffusion Path for Sampling
Omar Chehab, Anna Korba
TL;DR
This work tackles the challenge of sampling from target distributions known only up to a normalization constant, where score estimation is difficult for multimodal targets. It introduces the dilation path, a Dirac-proposal limit of the convolutional path, which yields closed-form score vectors and enables a simple adaptive Langevin sampler without Monte Carlo score estimation. The authors demonstrate the method's effectiveness on Gaussian mixtures and MNIST, showing improved mode coverage and practical sampling efficiency compared to traditional approaches. The dilation path thus provides a principled, computationally attractive route for diffusion-based sampling in unnormalized-density settings with multimodality, with potential for broad applicability in Bayesian inference and energy-based models.
Abstract
Diffusion models are state-of-the-art methods in generative modeling when samples from a target probability distribution are available, and can be efficiently sampled, using score matching to estimate score vectors guiding a Langevin process. However, in the setting where samples from the target are not available, e.g. when this target's density is known up to a normalization constant, the score estimation task is challenging. Previous approaches rely on Monte Carlo estimators that are either computationally heavy to implement or sample-inefficient. In this work, we propose a computationally attractive alternative, relying on the so-called dilation path, that yields score vectors that are available in closed-form. This path interpolates between a Dirac and the target distribution using a convolution. We propose a simple implementation of Langevin dynamics guided by the dilation path, using adaptive step-sizes. We illustrate the results of our sampling method on a range of tasks, and shows it performs better than classical alternatives.