AdaIR: Exploiting Underlying Similarities of Image Restoration Tasks with Adapters

TL;DR

AdaIR tackles the inefficiency of training separate restoration models for multiple degradations by introducing a shared foundation model augmented with lightweight adapters. A two-phase process first learns shareable representations through self-supervised pre-training on synthetic degradations, then adapts to specific tasks by training only task adapters while keeping the base model frozen. The approach achieves competitive performance with far fewer trainable parameters (≈1.9 MB) and reduced training time (≈7 hours per task), and it generalizes to unseen degradations better than fully task-specific baselines. Analyses of pre-training schemes provide guidance on how to select degradations for maximum transfer, highlighting AdaIR’s potential for scalable, multi-task image restoration.

Abstract

Existing image restoration approaches typically employ extensive networks specifically trained for designated degradations. Despite being effective, such methods inevitably entail considerable storage costs and computational overheads due to the reliance on task-specific networks. In this work, we go beyond this well-established framework and exploit the inherent commonalities among image restoration tasks. The primary objective is to identify components that are shareable across restoration tasks and augment the shared components with modules specifically trained for individual tasks. Towards this goal, we propose AdaIR, a novel framework that enables low storage cost and efficient training without sacrificing performance. Specifically, a generic restoration network is first constructed through self-supervised pre-training using synthetic degradations. Subsequent to the pre-training phase, adapters are trained to adapt the pre-trained network to specific degradations. AdaIR requires solely the training of lightweight, task-specific modules, ensuring a more efficient storage and training regimen. We have conducted extensive experiments to validate the effectiveness of AdaIR and analyze the influence of the pre-training strategy on discovering shareable components. Extensive experimental results show that AdaIR achieves outstanding results on multi-task restoration while utilizing significantly fewer parameters (1.9 MB) and less training time (7 hours) for each restoration task. The source codes and trained models will be released.

Figures (8)

• Figure 1: Comparison of different approaches. An illustration of different strategies to deal with multiple image restoration tasks. (a) Training separate models for each specific task, e.g., denoising, deraining, and supre-resolution. (b) Appending multiple heads and tails, which are respectively tailored to different tasks, on a shared backbone model. (c) Designing special blocks within the all-in-one model to encode and utilize degradation information without specifying the task explicitly. (d) Our proposed AdaIR exploiting task-specific adapters to address different restoration tasks.
• Figure 2: Overview. The proposed AdaIR framework. Our foundation model comprises feature extraction, pre-trained, and image restoration modules. The adapter modules interact with pre-trained modules to form adapter layers. When fine-tuning, the parameters of the foundation model are frozen; only the parameters in the adapter module are tunable.
• Figure 3: The qualitative results of $\text{Restormer}_{E+H}$restormer, $\text{PromptIR}_{E+H}$promptir, and our AdaIR in the denoising task with two noisy levels.
• Figure 4: The qualitative results of our AdaIR in the SR task with two upscaling factors.
• Figure 4: The average PSNR (dB) over the hard subsets of the restoration tasks including Gaussian denoising, super-resolution, Gaussian deblurring, and deraining. The best and second-best performing results are highlighted by the red and blue colors, respectively.