Stabilizing Diffusion Posterior Sampling by Noise--Frequency Continuation

TL;DR

A noise--frequency Continuation framework is proposed that constructs a continuous family of intermediate posteriors whose likelihood enforces measurement consistency only within a noise-dependent frequency band and is instantiated with a stabilized posterior sampler that combines a diffusion predictor, band-limited likelihood guidance, and a multi-resolution consistency strategy.

Abstract

Diffusion posterior sampling solves inverse problems by combining a pretrained diffusion prior with measurement-consistency guidance, but it often fails to recover fine details because measurement terms are applied in a manner that is weakly coupled to the diffusion noise level. At high noise, data-consistency gradients computed from inaccurate estimates can be geometrically incongruent with the posterior geometry, inducing early-step drift, spurious high-frequency artifacts, plus sensitivity to schedules and ill-conditioned operators. To address these concerns, we propose a noise--frequency Continuation framework that constructs a continuous family of intermediate posteriors whose likelihood enforces measurement consistency only within a noise-dependent frequency band. This principle is instantiated with a stabilized posterior sampler that combines a diffusion predictor, band-limited likelihood guidance, and a multi-resolution consistency strategy that aggressively commits reliable coarse corrections while conservatively adopting high-frequency details only when they become identifiable. Across super-resolution, inpainting, and deblurring, our method achieves state-of-the-art performance and improves motion deblurring PSNR by up to 5 dB over strong baselines.

Figures (9)

• Figure 1: Results of Different Sampling Steps. The first and third rows show the estimated image $\boldsymbol{\hat{x}}_0$, while the second and fourth rows show the corresponding noised image $\boldsymbol{x}_T$ at different timesteps. Across timesteps, the evaluated outputs preserve a similar coarse layout to the underlying $\boldsymbol{x}_0$. For example, this typically manifests as a centrally located face with a consistent surrounding background for the FFHQ dataset.
• Figure 2: Method Overview. First of all, a pre-trained diffusion model is applied to acquire the estimated $\boldsymbol{\hat{x}}_0$. Second, we optimize $\boldsymbol{\hat{x}}_0$ in frequency domain and apply Haar fusion. Afterwards, target features are generated after iteratively sampling, and accumulated errors and efficiency can be significantly improved.
• Figure 3: Qualitative results: the first four rows (a) present outputs on FFHQ $256\times 256$ dataset, and the last four rows (b) present outputs on ImageNet $256\times 256$ dataset. Our method performs better than the others in restoring both the coarse and fine-grained details of the input image, which reflect the robustness of our framework (We compared against methods that, as much as possible, produce better visual results and support a broader range of tasks).
• Figure 4: Details of the Low-pass Restoration Process. Results are evaluated on ImageNet and achieve a PSNR of 36.88 dB. (a) Masked spectrum magnitude maps induced by the noise scheduler. (b) Progressive low-pass reconstruction trajectory in the spatial domain controlled by a noise scheduler. (c) Sampling process.
• Figure 5: Details of Quantitative Indices Across Sampling Steps. The orange curves report metric values computed from the intermediate estimates $\boldsymbol{\hat{x}}_0$ produced by the optimization procedure in Eq. \ref{['ula_update']}, while the red curves report metric values computed from $\boldsymbol{\hat{x}}_0$ estimated by solvers. These results are evaluated on the FFHQ $256\times256$ dataset under the motion deblurring setting.

Theorems & Definitions (18)

• Lemma 1.1: Operator conditioning controls likelihood-gradient sensitivity
• proof
• Lemma 1.2: Large-$\sigma$ amplifies score errors into $\hat{\boldsymbol{x}}_0$ errors
• proof
• Theorem 1.3: Gradient mismatch amplification
• proof
• Theorem 1.4: Descent under inexact gradients
• proof
• Corollary 1.5: Band-limited likelihood reduces curvature and sensitivity
• Lemma 1.6: Non-expansiveness of spectral masking