DiracDiffusion: Denoising and Incremental Reconstruction with Assured Data-Consistency

TL;DR

Dirac proposes a diffusion-based inverse-problem solver that explicitly models the observed degraded measurement as a stochastic degradation process and then reverses this process to recover the clean signal while preserving data-consistency. The approach jointly learns a score network from degraded pairs, introduces an incremental reconstruction loss to stabilize reverse steps, and utilizes a degradation-scheduling strategy for controlled trade-offs between perceptual quality and distortion metrics. The framework yields state-of-the-art results on high-resolution deblurring and inpainting, with fast sampling and built-in data fidelity, validated on CelebA-HQ and ImageNet against strong baselines. A key limitation is the need to train a dedicated model per inverse problem, but the method offers flexible early-stopping and robust performance across test-time variations.

Abstract

Diffusion models have established new state of the art in a multitude of computer vision tasks, including image restoration. Diffusion-based inverse problem solvers generate reconstructions of exceptional visual quality from heavily corrupted measurements. However, in what is widely known as the perception-distortion trade-off, the price of perceptually appealing reconstructions is often paid in declined distortion metrics, such as PSNR. Distortion metrics measure faithfulness to the observation, a crucial requirement in inverse problems. In this work, we propose a novel framework for inverse problem solving, namely we assume that the observation comes from a stochastic degradation process that gradually degrades and noises the original clean image. We learn to reverse the degradation process in order to recover the clean image. Our technique maintains consistency with the original measurement throughout the reverse process, and allows for great flexibility in trading off perceptual quality for improved distortion metrics and sampling speedup via early-stopping. We demonstrate the efficiency of our method on different high-resolution datasets and inverse problems, achieving great improvements over other state-of-the-art diffusion-based methods with respect to both perceptual and distortion metrics.

Figures (17)

• Figure 1: Overview of our method: measurement acquisition is modeled as a gradual degradation and noising of an underlying clean ground truth signal via a Stochastic Degradation Process. We reconstruct the clean image from noisy measurements by learning to reverse the degradation process. Our technique allows for obtaining a variety of reconstructions with different perceptual quality-distortion trade-offs, all in a single sampling trajectory.
• Figure 2: Severity of degradations: We can always find a more degraded image $\bm{y}_{t"}$ from a less degraded version of the same clean image $\bm{y}_{t'}$ via the forward degradation transition function $\mathcal{G}_{t'\rightarrow t"}$, but not vice versa.
• Figure 3: Perception-distortion trade-off on CelebA-HQ deblurring: distortion metrics initially improve, peak fairly early in the reverse process, then gradually deteriorate, while perceptual metrics improve. We plot the mean of $30$ trajectories ($\pm std$ shaded) starting from the same measurement.
• Figure 4: Left: Data consistency in FFHQ inpainting. $\epsilon_{dc} := \left\lVert\bm{\tilde{y}} - \mathcal{A}_{1}(\hat{\bm{x}}_0(\bm{y}_t))\right\rVert^2$ measures how consistent is the clean image estimate with the measurement. We expect $\epsilon_{dc}$ to approach the noise floor $\sigma_1^2 = 0.0025$ in case of perfect data consistency. We plot $\bar{\epsilon}_{dc}$ the mean over the validation set. Dirac maintains data consistency throughout the reverse process. Right: Data consistency is not always achieved with DPS.
• Figure 4: Comparison with blending schedule on the FFHQ test split.

Theorems & Definitions (17)

• Definition 3.1: Severity of degradations
• Definition 3.2: Deterministic degradation process
• Definition 3.3: Stochastic degradation process (SDP)
• Theorem 3.4
• Definition 3.5: Data consistency
• Theorem 3.6: Data consistency over iterations
• Theorem A.1
• Lemma A.4
• proof
• Theorem 1