Denoising Diffusion Probabilistic Models in Six Simple Steps

TL;DR

The paper presents a concise, six-step framework for denoising diffusion probabilistic models (DDPMs) that reframes DDPMs as a sequence of simple supervised regression tasks generated by a Gaussian AR(1) augmentation. It develops two practical paths for training: a simplified objective with an epsilon-parameterisation and a variational diffusion objective based on SNR, while advocating parameter sharing (FiLM) and variance-reducing Gaussian denoisers to enable scalable training. It provides detailed derivations for the augmentation process, the conditional distributions, and the link to standard diffusion notation, and discusses the relationship—and limitations—of the ELBO perspective in this context. The notes also connect the DDPM design choices to denoising autoencoders and denoising score matching, offering practical guidance on schedule design and model parameterisation for robust, sample-efficient diffusion models.

Abstract

Denoising Diffusion Probabilistic Models (DDPMs) are a very popular class of deep generative model that have been successfully applied to a diverse range of problems including image and video generation, protein and material synthesis, weather forecasting, and neural surrogates of partial differential equations. Despite their ubiquity it is hard to find an introduction to DDPMs which is simple, comprehensive, clean and clear. The compact explanations necessary in research papers are not able to elucidate all of the different design steps taken to formulate the DDPM and the rationale of the steps that are presented is often omitted to save space. Moreover, the expositions are typically presented from the variational lower bound perspective which is unnecessary and arguably harmful as it obfuscates why the method is working and suggests generalisations that do not perform well in practice. On the other hand, perspectives that take the continuous time-limit are beautiful and general, but they have a high barrier-to-entry as they require background knowledge of stochastic differential equations and probability flow. In this note, we distill down the formulation of the DDPM into six simple steps each of which comes with a clear rationale. We assume that the reader is familiar with fundamental topics in machine learning including basic probabilistic modelling, Gaussian distributions, maximum likelihood estimation, and deep learning.

Figures (1)

• Figure 1: The augmented data at different fidelity levels (left), the data generation or denoising process used at test time (centre), and the augmentation process that generates the different fidelity levels via the noising process for training (right).