Restore Anything Model via Efficient Degradation Adaptation

TL;DR

RAM tackles the all-in-one image restoration problem by unifying degradations into a single, efficient model. It introduces Degradation Adaptation Blocks with a GatedDA mechanism that splits features into global-attention and local-gated paths, enabling a joint embedding of diverse degradations through an X-shaped fusion. Empirically, RAM achieves state-of-the-art results across three- and five-degradation benchmarks while reducing trainable parameters by about 82% and FLOPs by about 85% relative to the previous SOTA, demonstrating strong efficiency for edge devices. The approach offers a practical, scalable baseline for compact, unified restoration systems and provides a foundation for further improvements in degradation-aware imaging.

Abstract

With the proliferation of mobile devices, the need for an efficient model to restore any degraded image has become increasingly significant and impactful. Traditional approaches typically involve training dedicated models for each specific degradation, resulting in inefficiency and redundancy. More recent solutions either introduce additional modules to learn visual prompts significantly increasing model size or incorporate cross-modal transfer from large language models trained on vast datasets, adding complexity to the system architecture. In contrast, our approach, termed RAM, takes a unified path that leverages inherent similarities across various degradations to enable both efficient and comprehensive restoration through a joint embedding mechanism without scaling up the model or relying on large multimodal models. Specifically, we examine the sub-latent space of each input, identifying key components and reweighting them in a gated manner. This intrinsic degradation awareness is further combined with contextualized attention in an X-shaped framework, enhancing local-global interactions. Extensive benchmarking in an all-in-one restoration setting confirms RAM's SOTA performance, reducing model complexity by approximately 82% in trainable parameters and 85% in FLOPs. Our code and models will be publicly available.

Figures (8)

• Figure 1: Average PSNR and parameters comparison (i.e., our RAM archives 0.45 dB PSNR improvement with around 82% parameters reduced compared to the SOTA PromptIR potlapalli2023promptir)
• Figure 2: (a) Overall framework of the proposed RAM : a U-shaped structure which mainly consists of a convolutional patch embedding, a U-shape encoder-decoder main body, and an extra refined block. (b) Structure of each degradation adaptation block (DAB)
• Figure 3: Structure of our GatedDA. $\oplus$, $\textcircled{c}$, $\textcircled{g}$, and $\otimes$ denote the element-wise addition, channel-wise concatenation, GELU hendrycks2016gaussian activation, and element-wise multiplication, respectively.
• Figure 4: Visual comparison of RAM with state-of-the-art methods considering three degradations. Zoom in for a better view.
• Figure 5: Structure of different DAB variants.