DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior

TL;DR

DiffBIR addresses the challenge of blind image restoration by decoupling degradation removal from content regeneration, enabling a unified two-stage pipeline that leverages a diffusion-prior based generator with IRControlNet conditioning. A training-free region-adaptive restoration guidance further allows users to trade fidelity for realism during sampling. The approach achieves state-of-the-art results across blind image super-resolution, denoising, and face restoration on both synthetic and real-world datasets, demonstrating strong generalization and practical utility. While computationally intensive due to diffusion sampling, DiffBIR provides a versatile framework with broad applicability and a public code release.

Abstract

We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks in a unified framework. DiffBIR decouples blind image restoration problem into two stages: 1) degradation removal: removing image-independent content; 2) information regeneration: generating the lost image content. Each stage is developed independently but they work seamlessly in a cascaded manner. In the first stage, we use restoration modules to remove degradations and obtain high-fidelity restored results. For the second stage, we propose IRControlNet that leverages the generative ability of latent diffusion models to generate realistic details. Specifically, IRControlNet is trained based on specially produced condition images without distracting noisy content for stable generation performance. Moreover, we design a region-adaptive restoration guidance that can modify the denoising process during inference without model re-training, allowing users to balance realness and fidelity through a tunable guidance scale. Extensive experiments have demonstrated DiffBIR's superiority over state-of-the-art approaches for blind image super-resolution, blind face restoration and blind image denoising tasks on both synthetic and real-world datasets. The code is available at https://github.com/XPixelGroup/DiffBIR.

Figures (16)

• Figure 1: Comparisons of state-of-the-art methods and our DiffBIR for blind image super-resolution (BSR), blind image denoising (BID), and blind face restoration (BFR). (Zoom in for best view)
• Figure 2: The effects of condition information on generated results. The 2nd row shows that directly using LQ images as conditions causes unpleasant artifacts induced by different degradations (Gaussian, speckle, Poisson, and JPEG compression noises). While our DiffBIR's two-stage pipeline is more stable (see 3rd-row).
• Figure 3: The two-stage pipeline of DiffBIR. 1) Restoration Module (RM) for degradation removal; 2) Generation Module (GM) for realistic image reconstruction with optional region-adaptive restoration guidance for a trade-off between quality and fidelity.
• Figure 4: Architectures of our IRControlNet and four model variants.
• Figure 5: The training loss curves of IRControlNet and Variant 2,3,4 on ImageNet1k dataset under the same training setting.