Bridging Degradation Discrimination and Generation for Universal Image Restoration

TL;DR

BDG tackles universal image restoration by unifying degradation discrimination with generation priors through MAS-GLCM and a three-stage diffusion training. The MAS-GLCM descriptor provides fine-grained degradation identification, while the generation-bridging-restoration training preserves texture-rich generation without sacrificing fidelity. Key contributions include the MAS-GLCM characterization, the diffusion-forward formulation with conditioning, and the three-stage BDG training with dedicated losses to align degradation features to diffusion features and to enable robust real-world performance. Empirically, BDG achieves state-of-the-art results on all-in-one restoration, mixed degradations, and real-world super-resolution, demonstrating improved fidelity and preserved texture across diverse degradations.

Abstract

Universal image restoration is a critical task in low-level vision, requiring the model to remove various degradations from low-quality images to produce clean images with rich detail. The challenges lie in sampling the distribution of high-quality images and adjusting the outputs on the basis of the degradation. This paper presents a novel approach, Bridging Degradation discrimination and Generation (BDG), which aims to address these challenges concurrently. First, we propose the Multi-Angle and multi-Scale Gray Level Co-occurrence Matrix (MAS-GLCM) and demonstrate its effectiveness in performing fine-grained discrimination of degradation types and levels. Subsequently, we divide the diffusion training process into three distinct stages: generation, bridging, and restoration. The objective is to preserve the diffusion model's capability of restoring rich textures while simultaneously integrating the discriminative information from the MAS-GLCM into the restoration process. This enhances its proficiency in addressing multi-task and multi-degraded scenarios. Without changing the architecture, BDG achieves significant performance gains in all-in-one restoration and real-world super-resolution tasks, primarily evidenced by substantial improvements in fidelity without compromising perceptual quality. The code and pretrained models are provided in https://github.com/MILab-PKU/BDG.

Figures (9)

• Figure 1: (1) Visualization of MAS-GLCM in varying degradation levels. With an increase in degradation levels, the MAS-GLCM exhibits significant transformations. (2) The results of the T-SNE analysis for LQ images and MAS-GLCM across various degradation types demonstrate that MAS-GLCM possesses an enhanced capacity to distinguish between degradation types.
• Figure 2: Three training stages in BDG. (1) During the generation stage, the model focuses on obtaining generation priors. (2) In the bridging stage, the MAS-GLCM, which can identify degradation fine-grainedly, is aligned with the features of the pre-trained generation model, thereby endowing the model with initial capabilities in degradation discrimination. (3) In the restoration stage, the model is tasked with performing restoration.
• Figure 3: Visual comparison on the 5D all-in-one image restoration task. From top to bottom, each row corresponds to: deblurring, low-light enhancement, and deraining.
• Figure 4: Visual comparison on the real-world all-in-one image restoration task. From top to bottom, each row corresponds to: desnowing and low-light enhancement.
• Figure 5: MAS-GLCM on images with different texture.