Restorer: Removing Multi-Degradation with All-Axis Attention and Prompt Guidance

TL;DR

Restorer tackles the real-world challenge of multi-degradation image restoration with a single Transformer-based model. It introduces All-Axis Attention (AAA) to jointly model spatial and channel dependencies using stereo embedding, and employs textual degradation prompts to provide explicit task priors via CLIP, enabling iterative restoration of composite degradations without extra training. The approach yields strong, often state-of-the-art results across six restoration tasks and demonstrates robustness on real-world degradations with efficient inference. Ablation studies confirm the contributions of AAA, 3D-DCFFN, and textual prompts, supporting the practicality of Restorer for deployment in real-world scenarios.

Abstract

There are many excellent solutions in image restoration.However, most methods require on training separate models to restore images with different types of degradation.Although existing all-in-one models effectively address multiple types of degradation simultaneously, their performance in real-world scenarios is still constrained by the task confusion problem.In this work, we attempt to address this issue by introducing \textbf{Restorer}, a novel Transformer-based all-in-one image restoration model.To effectively address the complex degradation present in real-world images, we propose All-Axis Attention (AAA), a mechanism that simultaneously models long-range dependencies across both spatial and channel dimensions, capturing potential correlations along all axes.Additionally, we introduce textual prompts in Restorer to incorporate explicit task priors, enabling the removal of specific degradation types based on user instructions. By iterating over these prompts, Restorer can handle composite degradation in real-world scenarios without requiring additional training.Based on these designs, Restorer with one set of parameters demonstrates state-of-the-art performance in multiple image restoration tasks compared to existing all-in-one and even single-task models.Additionally, Restorer is efficient during inference, suggesting the potential in real-world applications.

Figures (23)

• Figure 1: Restored images of TransWeather and Restorer based on different textual prompts. TransWeather confuses the low-light enhancement task when performing the deblurring task, resulting in poor deblurring results. In contrast, Restorer at different textual prompts accurately performs the corresponding image restoration task.
• Figure 2: Illustration of spatial self-attention, channel self-attention, omni self-attention, and all-axis attention.
• Figure 3: The overall architecture of the proposed Restorer and the structure of each Restorer component. We utilize transposed convolution for upsampling and convolution with a stride of 2 for downsampling, respectively.
• Figure 4: Visualization of feature maps for each channel before and after the AAA module for different image restoration tasks. It can be observed that there are significant degradation residuals in the inter-channel feature maps before the AAA module, while the degradation residuals among the channels are eliminated after the AAA module.
• Figure 5: Visual comparison with SOTA image restoration algorithms in different image restoration tasks.