Simultaneous Image-to-Zero and Zero-to-Noise: Diffusion Models with Analytical Image Attenuation

TL;DR

This work introduces Diffusion Models with Analytical Image Attenuation (ADM), reimagining the forward diffusion as a dual path: image-to-zero via an analytic attenuation function $h_t$ and zero-to-noise via a Wiener process. A two-decoder network simultaneously predicts the clean image and noise, with a training objective that minimizes both components, enabling more efficient learning. The analyticity of the forward path yields a reverse-time sampling process that can use arbitrary step sizes, dramatically reducing function evaluations while maintaining image quality. ADM demonstrates competitive unconditioned generation with far fewer steps and state-of-the-art performance on several conditioned tasks (super-resolution, saliency, edge detection, inpainting) using as few as 5–10 steps. This approach offers a practical path to fast, high-fidelity diffusion generation and presents a framework for integrating analytic forward dynamics with learned reverse dynamics.

Abstract

Recent studies have demonstrated that the forward diffusion process is crucial for the effectiveness of diffusion models in terms of generative quality and sampling efficiency. We propose incorporating an analytical image attenuation process into the forward diffusion process for high-quality (un)conditioned image generation with significantly fewer denoising steps compared to the vanilla diffusion model requiring thousands of steps. In a nutshell, our method represents the forward image-to-noise mapping as simultaneous \textit{image-to-zero} mapping and \textit{zero-to-noise} mapping. Under this framework, we mathematically derive 1) the training objectives and 2) for the reverse time the sampling formula based on an analytical attenuation function which models image to zero mapping. The former enables our method to learn noise and image components simultaneously which simplifies learning. Importantly, because of the latter's analyticity in the \textit{zero-to-image} sampling function, we can avoid the ordinary differential equation-based accelerators and instead naturally perform sampling with an arbitrary step size. We have conducted extensive experiments on unconditioned image generation, \textit{e.g.}, CIFAR-10 and CelebA-HQ-256, and image-conditioned downstream tasks such as super-resolution, saliency detection, edge detection, and image inpainting. The proposed diffusion models achieve competitive generative quality with much fewer denoising steps compared to the state of the art, thus greatly accelerating the generation speed. In particular, to generate images of comparable quality, our models require only one-twentieth of the denoising steps compared to the baseline denoising diffusion probabilistic models. Moreover, we achieve state-of-the-art performances on the image-conditioned tasks using only no more than 10 steps.

Figures (11)

• Figure 1: High-quality images generated by the proposed ADMs under few-step settings. (a) 10-step unconditioned generation on the CelebA-HQ-256 dataset. (b) 5-step conditioned tasks including saliency detection, image inpainting, super-resolution and edge detection.
• Figure 2: Framework overview. (Top:) DPMs typically use an image-to-noise process, while we propose to replace it with two relatively simpler processes: the image-to-zero mapping and the zero-to-noise mapping. We use an analytic function (in blue boxes) to model image-to-zero, or the attenuation gradient of the image (red line in the middle image with a linear attenuation). The zero-to-noise path is governed by the standard Wiener process. (Bottom:) We compare the equations of forward sampling, reversed sampling, and training objectives of our method and DDPM. Note that $\mathbf{h}_{t}$ is the proposed analytical image attenuation function and $\boldsymbol{\phi}$ represents its hyperparameter.
• Figure 3: Comparing DDPM, its improved version, and ADM of image quality of unconditioned generation on CIFAR-10. (Left:) We evaluate image quality at each step using mean square error (MSE, solid line) and Fréchet inception distance (FID, dashed line) of the final $\mathbf{x}_{0}$ ($t=0$) and the estimated $\mathbf{\hat{x}}_{0}$ during denoising time. (Right:) Sample generated images from $t=0.4$ for the three compared methods. Improvement can be observed after adding to DDPM a branch that predicts $\mathbf{x}_0$. Further improvement is confirmed when the image-to-noise process is completely split to predict $\mathbf{x}_0$ and noise simultaneously.
• Figure 4: Architecture detail. The architecture is built upon U-Net. Different from the vanilla U-Net, we construct two decoders to parameterize $\boldsymbol{\phi}$ and $\boldsymbol{\epsilon}$ simultaneously. The 'Cond' represents the conditioned input and the conditional encoder is the general image backbone such as Swin (57)liu2021swin. The conditional branch is only used in conditioned generation tasks.
• Figure 5: Comparisons of 10-step unconditional generation on CIFAR10.