DAPS++: Rethinking Diffusion Inverse Problems with Decoupled Posterior Annealing

TL;DR

This work reframes diffusion-based inverse problems by showing that prior guidance is negligible in high-noise regimes, and proposes an EM-style, fully decoupled two-stage framework (DAPS++) that first initializes via diffusion priors and then performs likelihood-driven MCMC refinement within a constrained space. The E-step uses Tweedie-based denoising to initialize $\hat{\mathbf{x}}_0$ within the data manifold, followed by an M-step that updates $\mathbf{x}_0$ using a likelihood gradient and occasional re-noising to preserve diffusion characteristics. Theoretical Lipschitz analysis justifies neglecting the prior in time-marginal updates, and experiments on FFHQ and ImageNet show that DAPS++ achieves comparable or better reconstruction quality with substantially fewer neural function evaluations (NFEs), offering a scalable, principled approach for diffusion-based inverse problems. The results connect to and extend prior work (DPS, DAPS) by clarifying the separation between generation and data-consistency and enabling efficient, robust reconstruction across linear and nonlinear imaging tasks.

Abstract

From a Bayesian perspective, score-based diffusion solves inverse problems through joint inference, embedding the likelihood with the prior to guide the sampling process. However, this formulation fails to explain its practical behavior: the prior offers limited guidance, while reconstruction is largely driven by the measurement-consistency term, leading to an inference process that is effectively decoupled from the diffusion dynamics. To clarify this structure, we reinterpret the role of diffusion in inverse problem solving as an initialization stage within an expectation--maximization (EM)--style framework, where the diffusion stage and the data-driven refinement are fully decoupled. We introduce \textbf{DAPS++}, which allows the likelihood term to guide inference more directly while maintaining numerical stability and providing insight into why unified diffusion trajectories remain effective in practice. By requiring fewer function evaluations (NFEs) and measurement-optimization steps, \textbf{DAPS++} achieves high computational efficiency and robust reconstruction performance across diverse image restoration tasks.

Figures (11)

• Figure 1: (a) Evolution of the gradient ratio $\kappa_t$ with respect to noise level and its relative error during a Gaussian-blur iteration, illustrating that the data-consistency gradient dominates throughout the optimization process. (b) Comparison between DAPS results using the prior term and using only the data term across different numbers of annealing steps, showing that the prior contributes minimally and mainly shapes the time-marginal distribution.
• Figure 2: Diagram of the DAPS++ framework. The E-step provides the initial state and constructs the constrained optimization space $p(\mathbf{x}_0)$ for data-driven MCMC refinement, while the M-step optimizes within this space under measurement guidance. The two steps are fully decoupled, and after each M-step, controlled noise is re-injected by the diffusion process to initiate the next E-step.
• Figure 3: Qualitative results on representative inverse problems: (a) Gaussian blur reconstruction on the FFHQ-256 dataset compared with selected baselines. (b) Nonlinear blur reconstruction on ImageNet, comparing DAPS++-50 and DAPS++-100 alongside selected baselines. (c) High dynamic range (HDR) reconstruction on the FFHQ-256 dataset. (d) Motion blur reconstruction on the FFHQ-256 dataset. (e) Box inpainting results on ImageNet, where DAPS++ accurately restores structural details. (f) $8\times$ super-resolution using a pre-trained ImageNet diffusion model, demonstrating strong visual fidelity.
• Figure 4: Example under $\gamma{=}0.1$, $\sigma_{\min}{=}0.1$. Comparison between (a) DAPS++ and (b) DAPS for Gaussian blur shows that DAPS overfits noise, producing more artifacts in the final outputs.
• Figure 5: Evaluation of sampling time versus reconstruction quality. The x-axis denotes the per-image sampling time on an NVIDIA A100 (80GB PCIe) GPU, and the y-axis shows the LPIPS metric. Results are averaged over 100 FFHQ images for both the Gaussian blur and nonlinear blur tasks.