Reverse Diffusion Monte Carlo

TL;DR

A novel Monte Carlo sampling algorithm called reverse diffusion Monte Carlo (rdMC), which is distinct from the Markov chain Monte Carlo methods, and greatly improves over the Langevin-style MCMC sampling methods both theoretically and in practice.

Abstract

We propose a Monte Carlo sampler from the reverse diffusion process. Unlike the practice of diffusion models, where the intermediary updates -- the score functions -- are learned with a neural network, we transform the score matching problem into a mean estimation one. By estimating the means of the regularized posterior distributions, we derive a novel Monte Carlo sampling algorithm called reverse diffusion Monte Carlo (rdMC), which is distinct from the Markov chain Monte Carlo (MCMC) methods. We determine the sample size from the error tolerance and the properties of the posterior distribution to yield an algorithm that can approximately sample the target distribution with any desired accuracy. Additionally, we demonstrate and prove under suitable conditions that sampling with rdMC can be significantly faster than that with MCMC. For multi-modal target distributions such as those in Gaussian mixture models, rdMC greatly improves over the Langevin-style MCMC sampling methods both theoretically and in practice. The proposed rdMC method offers a new perspective and solution beyond classical MCMC algorithms for the challenging complex distributions.

Figures (16)

• Figure 1: Langevin dynamics (first row) versus reverse SDE (second row). The first and second rows depict the intermediate states of the Langevin algorithm and the reverse SDE, respectively, illustrating the transition from a standard normal $p_0$ to a Gaussian mixture $p_*$. It can be observed that due to the local nature of the information contained in $\nabla \ln p_*$, the Langevin algorithm tends to get stuck in modes close to the initializations. In contrast, the reverse SDE excels at transporting particles to different modes proportional to the target densities.
• Figure 2: Illustrations of $p_t$, $q_t$, and their log-Sobolev (LSI) constants. The target distribution $p_*$ is a Gaussian mixture. We choose $q_t(\cdot|x=0)$ for illustration. As $t$ increases, the modes of $p_t$ collapse to zero rapidly, corresponding to an improving LSI constant. For $q_t$, the barrier height of $q_t$ remains small, resulting in a relatively large LSI constant as long as $T=O(1)$. Thus initializing with $p_T$ and performing rdMC reduces computation complexity for multi-modal $p_*$.
• Figure 3: Maximum Mean Discrepancy (MMD) convergence of LMC, ULMC, rdMC. First row shows different target distributions, with increasing mode separation $r$ and barrier heights, leading to reduced log-Sobolev (LSI) constants. Second row displays the algorithms' convergence, revealing rdMC's pronounced advantage convergence compared to ULMC/LMC, especially for large $r$.
• Figure 4: Maximum Mean Discrepancy (MMD) convergence of LMC, ULMC, rdMC.
• Figure 5: Maximum Mean Discrepancy (MMD) convergence of LMC, ULMC, rdMC.

Theorems & Definitions (44)

• Lemma 1
• Theorem 1
• Lemma 2
• Proposition 1
• Lemma 3
• Proposition 2
• Definition 1
• Lemma 4
• proof
• Lemma 5