AWRaCLe: All-Weather Image Restoration using Visual In-Context Learning

TL;DR

AWRaCLe tackles all-weather image restoration by introducing visual in-context learning to AWIR, leveraging a degradation context pair and CLIP-based features. It proposes Degradation Context Extraction (DCE) to extract degradation-specific information and Context Fusion (CF) to inject this guidance into a Restormer backbone across multiple decoder levels, enabling robust, context-guided restoration. The approach achieves state-of-the-art performance on standard AWIR benchmarks and demonstrates the ability to selectively remove particular degradations by guiding the context, with strong robustness to context variation. This work highlights the practical potential of visual in-context learning for complex, multi-degradation restoration tasks and lays groundwork for broader deployment in real-world scenarios.

Abstract

All-Weather Image Restoration (AWIR) under adverse weather conditions is a challenging task due to the presence of different types of degradations. Prior research in this domain relies on extensive training data but lacks the utilization of additional contextual information for restoration guidance. Consequently, the performance of existing methods is limited by the degradation cues that are learnt from individual training samples. Recent advancements in visual in-context learning have introduced generalist models that are capable of addressing multiple computer vision tasks simultaneously by using the information present in the provided context as a prior. In this paper, we propose All-Weather Image Restoration using Visual In-Context Learning (AWRaCLe), a novel approach for AWIR that innovatively utilizes degradation-specific visual context information to steer the image restoration process. To achieve this, AWRaCLe incorporates Degradation Context Extraction (DCE) and Context Fusion (CF) to seamlessly integrate degradation-specific features from the context into an image restoration network. The proposed DCE and CF blocks leverage CLIP features and incorporate attention mechanisms to adeptly learn and fuse contextual information. These blocks are specifically designed for visual in-context learning under all-weather conditions and are crucial for effective context utilization. Through extensive experiments, we demonstrate the effectiveness of AWRaCLe for all-weather restoration and show that our method advances the state-of-the-art in AWIR.

Figures (5)

• Figure 1: Illustration of AWRaCLe: Our visual in-context learning approach for all-weather image restoration. The first two rows are the context pair. The third row is the query image that needs to be restored and the fourth row is our output. (d) and (e) show results for selective removal of haze and snow, respectively, from an image containing their mixture.
• Figure 2: Block diagram of the proposed visual in-context learning approach for AWIR. CLIP features are extracted from $I_{\texttt{d}}$ and $I_{\texttt{c}}$ which are subsequently fed to DCE blocks at different decoder levels, $l$. CF blocks then fuse the degradation information obtained from the DCE blocks with decoder features, $F^{l}$, from the query image $I_{\texttt{q}}$. Finally, the restored image is generated.
• Figure 3: Analysis of DCE block outputs.
• Figure 4: Comparison of activations of the restoration network prior to CF and after CF. Yellow-High, Blue-Low.
• Figure 5: Qualitative comparisons of AWRaCLe with top performing approaches (TSMC, PromptIR and DiffUIR) on SOTS, Rain100L, Rain100H and Snow100k datasets. Zoomed-in patches are provided for examining fine details.