Degradation-Aware All-in-One Image Restoration via Latent Prior Encoding

TL;DR

The paper addresses robust all-in-one image restoration under unknown, spatially varying degradations by reframing AIR as latent-prior inference. It introduces DAIR, a degradation-aware framework that learns latent priors from degraded images, employs a learnable degradation map for spatial localization, and uses a linear-cost 3WD decoder for restoration, guided by which/where/what reasoning. The approach yields state-of-the-art results across six common and five compound degradations, improves performance on unseen degradations, and enhances downstream tasks such as object detection, all with greater efficiency than prior methods. The proposed latent-prior paradigm eliminates reliance on manual prompts or handcrafted priors, offering strong generalization and practical applicability in real-world vision systems.

Abstract

Real-world images often suffer from spatially diverse degradations such as haze, rain, snow, and low-light, significantly impacting visual quality and downstream vision tasks. Existing all-in-one restoration (AIR) approaches either depend on external text prompts or embed hand-crafted architectural priors (e.g., frequency heuristics); both impose discrete, brittle assumptions that weaken generalization to unseen or mixed degradations. To address this limitation, we propose to reframe AIR as learned latent prior inference, where degradation-aware representations are automatically inferred from the input without explicit task cues. Based on latent priors, we formulate AIR as a structured reasoning paradigm: (1) which features to route (adaptive feature selection), (2) where to restore (spatial localization), and (3) what to restore (degradation semantics). We design a lightweight decoding module that efficiently leverages these latent encoded cues for spatially-adaptive restoration. Extensive experiments across six common degradation tasks, five compound settings, and previously unseen degradations demonstrate that our method outperforms state-of-the-art (SOTA) approaches, achieving an average PSNR improvement of 1.68 dB while being three times more efficient.

Figures (14)

• Figure 1: Comparison of common all-in-one image restoration paradigms. Existing approaches depend on explicit task-specific guidance through manual prompts or predefined architectural biases. Our DAIR learns degradation-aware representations directly from degraded images through latent prior inference.
• Figure 2: Overview of the proposed DAIR framework. (a) The overall architecture consists of an encoder-decoder structure complemented by degradation priors from a VAE encoder. The degraded image is encoded by our Latent Prior Encoding (LPE) in both luminance and chrominance spaces. (b) Latent Prior Encoding (which) comprises multiple Degradation Aware Encoder Blocks (DAEB) that encode degradation prior information from the VAE. Latents encoded at different stages are passed to the Degradation Map blocks. (c) The Degradation Mapping (where) block utilizes LPE latents and VAE priors to generate degradation maps for each stage. (d) The Latent Fusion (what) block combines luminance and chrominance latents to generate unified latents.
• Figure 3: Visual comparison for six common degradation settings under the all-in-one setting. The proposed method produces consistent and plausible images compared to the existing methods.
• Figure 4: Visual comparison for five compound degradations. The proposed DAIR can handle compound degradation and produce visually pleasing images.
• Figure 5: DAIR performance on unseen tasks: (a) Visual results for unseen compound degradation (low-light + rain); (b) t-SNE embeddings showing separation of unseen and known degradations.