Beyond Degradation Redundancy: Contrastive Prompt Learning for All-in-One Image Restoration

TL;DR

This work targets AiOIR by addressing two core weaknesses: prompt redundancy and misalignment with reconstruction objectives. It introduces Contrastive Prompt Learning (CPL), which combines a Sparse Prompt Module to foster task-specialized prompts with a sparse, input-driven routing mechanism and a Contrastive Prompt Regularization that enforces functional alignment between prompts and the restoration model via positive and negative prompt pairings. Across three-, five-, seven-task benchmarks, all-weather scenarios, real-world weather data, and composite degradations, CPL consistently improves over strong baselines and achieves state-of-the-art average performance, demonstrating robustness and cross-architecture applicability. The approach offers a practical, plug-in enhancement for AiOIR systems and suggests future directions in semantically aware negative sampling and more nuanced prompt–model interactions.

Abstract

All-in-One Image Restoration (AiOIR), which addresses diverse degradation types with a unified model, presents significant challenges in designing task-aware prompts that effectively guide restoration across multiple degradation scenarios. While adaptive prompt learning enables end-to-end optimization, it often yields overlapping or redundant task representations. Conversely, explicit prompts derived from pretrained classifiers enhance discriminability but discard critical visual information needed for reconstruction. To address these limitations, we introduce Contrastive Prompt Learning (CPL), a framework that aims to improve prompt-task alignment through two complementary components: a Sparse Prompt Module (SPM) that efficiently captures degradation-aware representations while reducing redundancy, and a Contrastive Prompt Regularization (CPR) that explicitly strengthens task boundaries by incorporating negative prompt samples across different degradation types. Unlike previous approaches that focus primarily on degradation classification, CPL directly optimizes the interaction between prompts and the restoration model. Extensive experiments across five benchmarks show that CPL consistently boosts the performance of strong AiOIR baselines across diverse scenarios. Our approach achieves state-of-the-art average performance on these benchmarks, providing a general and robust solution for AiOIR. The code is available at https://github.com/Aitical/CPLIR

Figures (11)

• Figure 1: Comparison of prompt-based AiOIR frameworks. (a) Adaptive end-to-end learnable prompts. (b) Two-stage frameworks with pretrained degradation encoders. (c) The proposed CPL paradigm with sparse prompt selection and CPR.
• Figure 2: Performance comparison. Integrating our CPL framework into existing all-in-one models improves performance across various tasks in our experiments.
• Figure 3: Illustration of the proposed CPL framework with the SPM and CPR. We adopt a stacked Transformer block (T-Block) backbone following Potlapalli2023promptir. Red dashed arrows denote the generation of negative samples used only during training.
• Figure 4: Comparison of prompt selection probability distributions between baseline PromptIR Potlapalli2023promptir and CPL across different degradation tasks. The x-axis (0--4) denotes prompt indices and the y-axis denotes selection probabilities. PromptIR (blue) activates multiple prompts for a given task, whereas CPL (red) produces more concentrated, task-specific prompt selections.
• Figure 5: Residual analysis for CPR. PSNR values are shown in the top-right corner of each image. Top row: haze example. Bottom row: rain example. Residual maps highlight differences between reconstructions obtained with matched and mismatched prompts.