Manifold-aware Representation Learning for Degradation-agnostic Image Restoration

TL;DR

The paper addresses all-in-one image restoration under diverse, mixed degradations by introducing MIRAGE, a lightweight backbone that explicitly splits features into three parallel branches—attention for global context, convolution for local structure, and MLP for channel-wise statistics—via Mixed Degradation Adaptation Blocks. It couples this architecture with a cross-layer SPD manifold contrastive learning objective that aligns shallow and latent representations through second-order statistics, implemented as an SPD-embedded contrastive loss $\,\mathcal{L}_{SPD}$ alongside $L_1$ and a real-valued Fourier loss. Experiments demonstrate state-of-the-art performance across All-in-One, mixed, and zero-shot settings, including highly efficient Tiny ($6$M) and Small ($10$M) variants with favorable PSNR/SSIM and FLOPs, underscoring strong generalization and efficiency. The work offers a scalable path for degradation-agnostic restoration and lays groundwork for extensions to video and multi-modal tasks, leveraging principled geometry-aware representation learning.

Abstract

Image Restoration (IR) aims to recover high quality images from degraded inputs affected by various corruptions such as noise, blur, haze, rain, and low light conditions. Despite recent advances, most existing approaches treat IR as a direct mapping problem, relying on shared representations across degradation types without modeling their structural diversity. In this work, we present MIRAGE, a unified and lightweight framework for all in one IR that explicitly decomposes the input feature space into three semantically aligned parallel branches, each processed by a specialized module attention for global context, convolution for local textures, and MLP for channel-wise statistics. This modular decomposition significantly improves generalization and efficiency across diverse degradations. Furthermore, we introduce a cross layer contrastive learning scheme that aligns shallow and latent features to enhance the discriminability of shared representations. To better capture the underlying geometry of feature representations, we perform contrastive learning in a Symmetric Positive Definite (SPD) manifold space rather than the conventional Euclidean space. Extensive experiments show that MIRAGE not only achieves new state of the art performance across a variety of degradation types but also offers a scalable solution for challenging all-in-one IR scenarios. Our code and models will be publicly available at https://amazingren.github.io/MIRAGE/.

Figures (11)

• Figure 1: (a)-(d): Visual comparison for Denoising, Deraining, Composited Degradations (low-light, haze, and snow), and underwater image enhancement. (e): The average PSNR and SSIM comparison across 4 challenging all-in-one and 1 zero-shot settings (Please zoom in for a better view).
• Figure 2: (a)-(c): The most adopted all-in-one image restoration encodedr-decoder pipelines. (d): The toy illustration of our SPD contrastive pipeline. (e): The overall framework of the proposed MIRAGE : i.e., a convolutional patch embedding, a U-shape encoder-decoder main body, an extra refined block, and the proposed SPD contrastive learning algorithm. (f): Structure of each mixed degradation adaptation block (MDAB).
• Figure 3: Channel redundancy analysis across multiple feature scales. (a) Cumulative explained variance curves from PCA applied to the channel dimension of features from 1-4 scales and one latent scale. (b) Normalized singular value spectra (in log scale) of the same features via SVD. Latent feature in both plots means the channel-wise projected 4th Scale feature.
• Figure 4: (a)-(d): The channel-wise similarity matrix from the 1st Scale ($H \times W \times C$) to the 4th Scale ($H/8 \times W/8 \times 8C$). (e): The channel-wise similarity matrix of (d) after channel-wise projection.
• Figure A: The illustration of different designs of the proposed MDAB.