DDIM Redux: Mathematical Foundation and Some Extension

TL;DR

Addresses the theoretical foundations of diffusion-based generative models by deriving exact backward trajectory in the probability-flow ODE and exact covariance ${\bm \Sigma}(t)$ for gDDIM under a linear SDE $d{\bf x} = {\bf f}(t){\bf x}dt + {\bf g}(t)d{\bf w}$. It reframes the diffusion/noising process as an approach to equilibrium and clarifies the efficiency of the exponential-integrator scheme via a change of variables to ${\bf y}$. It introduces paDDIM by a principal-axis decomposition ${\bm \Sigma}(t)=V(t)V^T(t)$ and per-axis dynamics, and shows conditions under which DDIM yields no circulating current (${\bf Q}=0$). The work tightens the mathematical foundations of DDIM/gDDIM, provides exact tools for analysis, and suggests practical paths to faster, more stable generation through paDDIM.

Abstract

This note provides a critical review of the mathematical concepts underlying the generalized diffusion denoising implicit model (gDDIM) and the exponential integrator (EI) scheme. We present enhanced mathematical results, including an exact expression for the reverse trajectory in the probability flow ODE and an exact expression for the covariance matrix in the gDDIM scheme. Furthermore, we offer an improved understanding of the EI scheme's efficiency in terms of the change of variables. The noising process in DDIM is analyzed from the perspective of non-equilibrium statistical physics. Additionally, we propose a new scheme for DDIM, called the principal-axis DDIM (paDDIM).