Advances in Diffusion-Based Generative Compression

TL;DR

Advances in Diffusion-Based Generative Compression surveys how diffusion and related generative models enable high-realism reconstructions at extremely low bit-rates through a two-stage compress-then-refine framework. It analyzes deterministic (no common randomness) and stochastic (with common randomness) architectures under rate-distortion-perception theory, and reviews representative methods such as CDC, HFD, and LDM-based approaches, alongside Gaussian diffusion and progressive coding via channel simulation. The paper connects diffusion-based compression to inverse problems, highlights practical implementations like Dithered Quantization, and discusses computational demands, evaluation challenges, and open research directions for improving efficiency and bridging theory with practice. It also foregrounds the role of common randomness in surpassing conventional rate-distortion limits and outlines progressive coding strategies that enable streaming-compatible reconstructions. Overall, the work consolidates a unified framework for diffusion-based neural compression and identifies key pathways to scalable, perceptually realistic codecs.

Abstract

Popularized by their strong image generation performance, diffusion and related methods for generative modeling have found widespread success in visual media applications. In particular, diffusion methods have enabled new approaches to data compression, where realistic reconstructions can be generated at extremely low bit-rates. This article provides a unifying review of recent diffusion-based methods for generative lossy compression, with a focus on image compression. These methods generally encode the source into an embedding and employ a diffusion model to iteratively refine it in the decoding procedure, such that the final reconstruction approximately follows the ground truth data distribution. The embedding can take various forms and is typically transmitted via an auxiliary entropy model, and recent methods also explore the use of diffusion models themselves for information transmission via channel simulation. We review representative approaches through the lens of rate-distortion-perception theory, highlighting the role of common randomness and connections to inverse problems, and identify open challenges.

Figures (1)

• Figure 1: Common architecture of diffusion-based compression algorithms considered in this article. They follow a two-stage "compress-then-reconstruct" approach. In stage 1 (illustrated by the box on the left), an embedding $Y$ of the source $X$ is computed and transmitted from the sender to the receiver. Its behavior can be described by a transition kernel $P_{Y|X}$, which is simulated by algorithms that exploit a common source of randomness ($W$). For algorithms based on nonlinear transform coding, this kernel is a deterministic function. In stage 2 (illustrated by the box on the right), the receiver performs additional computation to estimate $\hat{X}$ from $Y$, which can be viewed as solving an inverse problem given a fixed corruption process $P_{Y|X}$. This is commonly done by feeding $Y$ into a (conditional) diffusion/flow-matching model $v_\theta(x_t, y, t)$ and solving a corresponding ODE or SDE. The performance of the algorithm is evaluated based on the expected bit-rate $\mathop{\mathrm{\mathbb{E}}}\nolimits[|M|]$, distortion $\mathop{\mathrm{\mathbb{E}}}\nolimits[\rho(X, \hat{X})]$, and divergence (realism, or rather the lack thereof) $\texttt{d}(P_X, P_{\hat{X}})$.