Taming Generative Diffusion Prior for Universal Blind Image Restoration

TL;DR

The BIR-D is a generative diffusion prior for universal blind image restoration dubbed BIR-D, which utilizes an optimizable convolutional kernel to simulate the degradation model and dynamically update the parameters of the kernel in the diffusion steps, enabling it to achieve blind image restoration results even in various complex situations.

Abstract

Diffusion models have been widely utilized for image restoration. However, previous blind image restoration methods still need to assume the type of degradation model while leaving the parameters to be optimized, limiting their real-world applications. Therefore, we aim to tame generative diffusion prior for universal blind image restoration dubbed BIR-D, which utilizes an optimizable convolutional kernel to simulate the degradation model and dynamically update the parameters of the kernel in the diffusion steps, enabling it to achieve blind image restoration results even in various complex situations. Besides, based on mathematical reasoning, we have provided an empirical formula for the chosen of adaptive guidance scale, eliminating the need for a grid search for the optimal parameter. Experimentally, Our BIR-D has demonstrated superior practicality and versatility than off-the-shelf unsupervised methods across various tasks both on real-world and synthetic datasets, qualitatively and quantitatively. BIR-D is able to fulfill multi-guidance blind image restoration. Moreover, BIR-D can also restore images that undergo multiple and complicated degradations, demonstrating the practical applications.

Figures (12)

• Figure 1: Blind Image Restoration Diffusion Model (BIR-D) can achieve high-quality restoration for different types of degraded images. BIR-D not only has the capability to restore (a) linear inverse problems when the degradation function is known. BIR-D can also achieve high-quality image restoration in (b) blind issues with unknown degradation functions, as well as in (c) mixed degradation and real degradation scenarios.
• Figure 2: Overview of BIR-D. Degraded image $y$ was given during the sampling process. BIR-D systematically incorporates guidance from degraded images in the reverse process of the diffusion model and optimizes the degraded model at the same time. For degraded image $y$, pre-training is first performed to provide a better initial state for BIR-D. BIR-D introduces a distance function in each step of the reverse process of the diffusion model to describe the distance loss between the degraded image $y$ and the generated image $\tilde{x}_0$ after the degradation function, so that the gradient could be used to update and simulate a better degradation function. Based on the empirical formula, the adaptive guidance scale can be calculated to provide optimal guidance during the sampling process.
• Figure 3: Comparison of image quality for blind face restoration results on LFW wang2021towards and WIDER dataset zhou2022towards.
• Figure 4: Comparison of colorization image on ImageNet 1kpan2021exploiting. BIR-D can generate various outputs on the same input image.
• Figure 5: Results of linear degradation tasks on 256 × 256 images from ImageNet 1k.