Robust Posterior Diffusion-based Sampling via Adaptive Guidance Scale

TL;DR

AdaPS addresses the challenge of balancing the diffusion prior with data fidelity in posterior sampling for inverse problems by introducing a hyperparameter-free, adaptive guidance scale. The method reformulates DDIM for conditional sampling and uses an alignment-based update that combines a Jacobian-free magnitude estimate with a potentially Jacobian-aware direction, via a MAP-based surrogate. This yields robust, perceptually superior reconstructions across tasks like super-resolution and deblurring, achieving best or near-best LPIPS with minimal PSNR loss and demonstrating stability as the number of diffusion steps changes. The approach scales naturally with the diffusion schedule and stochasticity, obviating task-specific hyperparameter tuning and offering a practical, efficient pathway for diffusion priors in real-world inverse problems.

Abstract

Diffusion models have recently emerged as powerful generative priors for solving inverse problems, achieving state-of-the-art results across various imaging tasks. A central challenge in this setting lies in balancing the contribution of the prior with the data fidelity term: overly aggressive likelihood updates may introduce artifacts, while conservative updates can slow convergence or yield suboptimal reconstructions. In this work, we propose an adaptive likelihood step-size strategy to guide the diffusion process for inverse-problem formulations. Specifically, we develop an observation-dependent weighting scheme based on the agreement between two different approximations of the intractable intermediate likelihood gradients, that adapts naturally to the diffusion schedule, time re-spacing, and injected stochasticity. The resulting approach, Adaptive Posterior diffusion Sampling (AdaPS), is hyperparameter-free and improves reconstruction quality across diverse imaging tasks - including super-resolution, Gaussian deblurring, and motion deblurring - on CelebA-HQ and ImageNet-256 validation sets. AdaPS consistently surpasses existing diffusion-based baselines in perceptual quality with minimal or no loss in distortion, without any task-specific tuning. Extensive ablation studies further demonstrate its robustness to the number of diffusion steps, observation noise levels, and varying stochasticity.

Figures (9)

• Figure 1: Results of our method AdaPS compared to selected baselines across several noisy image reconstruction tasks. Despite being simple and hyperparameter-free, AdaPS consistently balances distortion and perceptual quality without any task-specific tuning, particularly under realistic noise levels. On ImageNet-256, AdaPS improves LPIPS over DDPG with almost no PSNR cost, and on CelebA-HQ it yields substantial perceptual gains with only minimal PSNR reduction. Additional results are detailed in Section \ref{['sec:exp']}.
• Figure 2: Schematic overview of our method. We introduce a principled, hyperparameter-free rule for balancing likelihood guidance with the prior in diffusion-based posterior sampling.
• Figure 3: Qualitative comparison of AdaPS and representative methods. Best viewed in zoom-in.
• Figure 4: Quantitative comparison. AdaPS scales with the number of steps, while $\Pi$GDM does not.
• Figure 5: Qualitative comparison. $\Pi$GDM deteriorates at larger step counts, while AdaPS remains stable and continues improving.

Theorems & Definitions (2)

• Remark 1
• Remark 2