Neural Degradation Representation Learning for All-In-One Image Restoration

TL;DR

This work tackles the problem of unknown, diverse image degradations by learning Neural Degradation Representation (NDR) within a single all-in-one restoration network, NDR-Restore. It introduces two modules, Degradation Query (DQ) and Degradation Injection (DI), to approximate and inject degradation information via a low-rank CP-based fusion, and trains the system with a bidirectional optimization against an auxiliary NDR-Degrad to ensure the degradation latent meaningfully captures degradations. The approach demonstrates strong performance across denoising, deraining, dehazing, and super-resolution, as well as multi-degradation scenarios and real-captured images, with ablations validating the contributions of NDR, DQ, and DI. The results indicate practical potential for robust restoration under unknown real-world degradations with competitive efficiency and stable cross-task behavior.

Abstract

Existing methods have demonstrated effective performance on a single degradation type. In practical applications, however, the degradation is often unknown, and the mismatch between the model and the degradation will result in a severe performance drop. In this paper, we propose an all-in-one image restoration network that tackles multiple degradations. Due to the heterogeneous nature of different types of degradations, it is difficult to process multiple degradations in a single network. To this end, we propose to learn a neural degradation representation (NDR) that captures the underlying characteristics of various degradations. The learned NDR decomposes different types of degradations adaptively, similar to a neural dictionary that represents basic degradation components. Subsequently, we develop a degradation query module and a degradation injection module to effectively recognize and utilize the specific degradation based on NDR, enabling the all-in-one restoration ability for multiple degradations. Moreover, we propose a bidirectional optimization strategy to effectively drive NDR to learn the degradation representation by optimizing the degradation and restoration processes alternately. Comprehensive experiments on representative types of degradations (including noise, haze, rain, and downsampling) demonstrate the effectiveness and generalization capability of our method.

Figures (16)

• Figure 1: Comparison between our method and other methods. zamir2022restormerchen2021pre fail to restore clean image if the model mismatches the degradation. Our method handles multiple degradations with a single network and produces more visually appealing results than the existing two-stage all-in-one model li2022all.
• Figure 2: Overview of our method. We construct NDR-Restore using a multi-scale architecture. NDR-Restore utilizes the DQ module to approach degradation and leverages the DI module to facilitate the interaction between degradation information and image features. NDR captures the underlying characteristics of multiple degradations and is utilized in the DQ module to generate degradation.
• Figure 3: Network details. (a) The implementation of the encoder and decoder in NDR-Restore. It takes transformer-based attention mechanism zamir2022restormer to extract shallow and deep features. (b) The details of CP-Conv, referred to Eq. \ref{['eq:cp1']} & \ref{['eq:cp2']}.
• Figure 4: Visualizations of NDR $D$, affinity matrices $S$, and approximated degradations $U$. Please note in $S$. We can observe distinguishing activation of different degradations and similar activation of the same degradation, which demonstrates the effective degradation approximation of DQ module and degradation representation of NDR.
• Figure 5: Features processed by the DI module. The residual features (before and after the DI module) only contain the degradation information, which demonstrates the effective degradation removal of the DI module.