Image Restoration by Denoising Diffusion Models with Iteratively Preconditioned Guidance
Tomer Garber, Tom Tirer
TL;DR
This work tackles image restoration under a linear observation model $\mathbf{y}=\mathbf{A}\mathbf{x}^*+\mathbf{e}$ by introducing Iterative Denoising and Preconditioned Guidance (IDPG) and its diffusion-based sampler variant DDPG. The core idea is to interpolate data-fidelity guidance between back-projection and least-squares steps via an iteration-dependent preconditioner $\mathbf{W}_t=(1-\delta_t)(\mathbf{A}\mathbf{A}^T+\eta\mathbf{I})^{-1}+\delta_t c\mathbf{I}_m$, yielding guidance $\mathbf{g}_{\delta_t}$ that balances data-consistency and noise robustness. Theoretical analysis shows BP, WLS, and LS induce distinct bias and variance profiles, justifying a gradual transition from BP to LS to achieve faster early convergence while controlling noise amplification. Empirically, IDPG improves over IDBP and LS-based baselines, and DDPG delivers superior perceptual quality (lower LPIPS) with competitive PSNR across deblurring and super-resolution tasks on CelebA-HQ and ImageNet, including noisy and non-separable degradations; the approach remains SVD-free and computationally efficient thanks to FFT-based or CG-based inverses. Overall, the method provides a flexible, robust restoration framework that integrates pretrained denoisers with data-fidelity guidance, with potential extensions to learn the preconditioners $\{\mathbf{W}_t\}$.
Abstract
Training deep neural networks has become a common approach for addressing image restoration problems. An alternative for training a "task-specific" network for each observation model is to use pretrained deep denoisers for imposing only the signal's prior within iterative algorithms, without additional training. Recently, a sampling-based variant of this approach has become popular with the rise of diffusion/score-based generative models. Using denoisers for general purpose restoration requires guiding the iterations to ensure agreement of the signal with the observations. In low-noise settings, guidance that is based on back-projection (BP) has been shown to be a promising strategy (used recently also under the names "pseudoinverse" or "range/null-space" guidance). However, the presence of noise in the observations hinders the gains from this approach. In this paper, we propose a novel guidance technique, based on preconditioning that allows traversing from BP-based guidance to least squares based guidance along the restoration scheme. The proposed approach is robust to noise while still having much simpler implementation than alternative methods (e.g., it does not require SVD or a large number of iterations). We use it within both an optimization scheme and a sampling-based scheme, and demonstrate its advantages over existing methods for image deblurring and super-resolution.