Mixed Degradation Image Restoration via Local Dynamic Optimization and Conditional Embedding

TL;DR

A novel multiple-in-one IR model that can effectively restore images with both single and mixed degradations is proposed and an efficient Conditional Feature Embedding module is developed that guides the decoder in leveraging degradation-type-related features, significantly improving the model's performance in mixed degradation restoration scenarios.

Abstract

Multiple-in-one image restoration (IR) has made significant progress, aiming to handle all types of single degraded image restoration with a single model. However, in real-world scenarios, images often suffer from combinations of multiple degradation factors. Existing multiple-in-one IR models encounter challenges related to degradation diversity and prompt singularity when addressing this issue. In this paper, we propose a novel multiple-in-one IR model that can effectively restore images with both single and mixed degradations. To address degradation diversity, we design a Local Dynamic Optimization (LDO) module which dynamically processes degraded areas of varying types and granularities. To tackle the prompt singularity issue, we develop an efficient Conditional Feature Embedding (CFE) module that guides the decoder in leveraging degradation-type-related features, significantly improving the model's performance in mixed degradation restoration scenarios. To validate the effectiveness of our model, we introduce a new dataset containing both single and mixed degradation elements. Experimental results demonstrate that our proposed model achieves state-of-the-art (SOTA) performance not only on mixed degradation tasks but also on classic single-task restoration benchmarks.

Figures (3)

• Figure 1: (a) Models for specific single-degradation tasks require multiple models and training for different single degradations. (b) A general model for single-degradation restoration needs multiple training sessions for various single degradations. (c) A multi-task model for single-degradation restoration requires one model and one training session but lacks design for mixed-degradation scenarios. (d) Our proposed scenario is one in which a single model and training session can handle both single-degradation and mixed-degradation images.
• Figure 2: (a) illustrates the comprehensive architectural schematic of our proposed model, MDIR, adhering to the encoder-decoder paradigm. The encoding phase encompasses three sequential encoding blocks, while the decoding phase comprises three decoding blocks along with three instances of the Conditional Feature Embedding Module (CFE). The configuration of each encoding and decoding block is delineated in (c), showcasing an assembly of multiple residual blocks succeeded by the LDO-cored structure (d). Sub-figure (b) denotes the CFE, which facilitates the integration between the classifier and the primary restoration network.
• Figure 3: The results of various methods for mixed degradation image restoration.