Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diffusion annealed Langevin dynamics: a theoretical study

Patrick Cattiaux, Paula Cordero-Encinar, Arnaud Guillin

TL;DR

This paper builds a rigorous theoretical foundation for diffusion annealed Langevin dynamics, a score-based diffusion that interpolates between a base distribution and a target. It proves existence and uniqueness of solutions and provides quantitative control of the KL bias between the annealed process and the target time-marginals, using Poincaré inequalities and their perturbations. It then strengthens the analysis by introducing perturbation-based bounds for the Poincaré constant and, crucially, leveraging logarithmic Sobolev inequalities to achieve sharper convergence and bias estimates, including potential dimension-free behavior in certain settings. The results extend applicability to heavy-tailed bases and furnish a principled, inequality-driven framework for assessing sampling efficiency in score-based diffusion models. Overall, the work connects Nelson processes, functional inequalities, and entropic transport to provide solid theoretical guarantees for diffusion-annealed samplers.

Abstract

In this work we study the diffusion annealed Langevin dynamics, a score-based diffusion process recently introduced in the theory of generative models and which is an alternative to the classical overdamped Langevin diffusion. Our goal is to provide a rigorous construction and to study the theoretical efficiency of these models for general base distribution as well as target distribution. As a matter of fact these diffusion processes are a particular case of Nelson processes i.e. diffusion processes with a given flow of time marginals. Providing existence and uniqueness of the solution to the annealed Langevin diffusion leads to proving a Poincaré inequality for the conditional distribution of $X$ knowing $X+Z=y$ uniformly in $y$, as recently observed by one of us and her coauthors. Part of this work is thus devoted to the study of such Poincaré inequalities. Additionally we show that strengthening the Poincaré inequality into a logarithmic Sobolev inequality improves the efficiency of the model.

Diffusion annealed Langevin dynamics: a theoretical study

TL;DR

This paper builds a rigorous theoretical foundation for diffusion annealed Langevin dynamics, a score-based diffusion that interpolates between a base distribution and a target. It proves existence and uniqueness of solutions and provides quantitative control of the KL bias between the annealed process and the target time-marginals, using Poincaré inequalities and their perturbations. It then strengthens the analysis by introducing perturbation-based bounds for the Poincaré constant and, crucially, leveraging logarithmic Sobolev inequalities to achieve sharper convergence and bias estimates, including potential dimension-free behavior in certain settings. The results extend applicability to heavy-tailed bases and furnish a principled, inequality-driven framework for assessing sampling efficiency in score-based diffusion models. Overall, the work connects Nelson processes, functional inequalities, and entropic transport to provide solid theoretical guarantees for diffusion-annealed samplers.

Abstract

In this work we study the diffusion annealed Langevin dynamics, a score-based diffusion process recently introduced in the theory of generative models and which is an alternative to the classical overdamped Langevin diffusion. Our goal is to provide a rigorous construction and to study the theoretical efficiency of these models for general base distribution as well as target distribution. As a matter of fact these diffusion processes are a particular case of Nelson processes i.e. diffusion processes with a given flow of time marginals. Providing existence and uniqueness of the solution to the annealed Langevin diffusion leads to proving a Poincaré inequality for the conditional distribution of knowing uniformly in , as recently observed by one of us and her coauthors. Part of this work is thus devoted to the study of such Poincaré inequalities. Additionally we show that strengthening the Poincaré inequality into a logarithmic Sobolev inequality improves the efficiency of the model.

Paper Structure

This paper contains 11 sections, 25 theorems, 149 equations.

Table of Contents

  1. Introduction
  2. Score based Stochastic dynamics
  3. Generative diffusion model
  4. Annealed Langevin process
  5. When functional inequalities enter the game, or not
  6. About the Poincaré constant $C_P(t)$
  7. General properties of the Poincaré inequality
  8. Convexity and Poincaré inequality
  9. Perturbation of Poincaré inequality
  10. New perturbation results for the Poincaré constant
  11. Using logarithmic Sobolev inequalities

Key Result

Theorem 2.1

Assume that for all $t \in [0,T]$, $\int |x|^2 \, p_t(x) \, dx < +\infty$. Define, when it exists, where $W_2$ denotes the $2$-Wasserstein distance. Assume that $|p'_.| \in \mathbb L^1([0,T])$. Remark that $|(\hat{p})'_t|=\kappa \, |p'_{\kappa t}|$ for all $t \in [0,T/\kappa]$. Then there exists a Borel vector field $v_.$ such that eqtransport is satisfied and

Theorems & Definitions (58)

  • Theorem 2.1
  • Theorem 2.2
  • proof
  • Remark 2.3
  • Corollary 2.4
  • proof
  • Remark 2.5
  • Remark 2.6
  • Remark 2.7
  • Proposition 3.1
  • ...and 48 more