CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration

TL;DR

Compound Adverse Weather Mamba is proposed, the first end-to-end framework that jointly performs image fusion and compound weather restoration with unified shared weights and results excel in downstream tasks covering semantic segmentation and object detection, confirming the practical value in real-world adverse weather perception.

Abstract

Multimodal Image Fusion (MMIF) integrates complementary information from various modalities to produce clearer and more informative fused images. MMIF under adverse weather is particularly crucial in autonomous driving and UAV monitoring applications. However, existing adverse weather fusion methods generally only tackle single types of degradation such as haze, rain, or snow, and fail when multiple degradations coexist (e.g., haze+rain, rain+snow). To address this challenge, we propose Compound Adverse Weather Mamba (CAWM-Mamba), the first end-to-end framework that jointly performs image fusion and compound weather restoration with unified shared weights. Our network contains three key components: (1) a Weather-Aware Preprocess Module (WAPM) to enhance degraded visible features and extracts global weather embeddings; (2) a Cross-modal Feature Interaction Module (CFIM) to facilitate the alignment of heterogeneous modalities and exchange of complementary features across modalities; and (3) a Wavelet Space State Block (WSSB) that leverages wavelet-domain decomposition to decouple multi-frequency degradations. WSSB includes Freq-SSM, a module that models anisotropic high-frequency degradation without redundancy, and a unified degradation representation mechanism to further improve generalization across complex compound weather conditions. Extensive experiments on the AWMM-100K benchmark and three standard fusion datasets demonstrate that CAWM-Mamba consistently outperforms state-of-the-art methods in both compound and single-weather scenarios. In addition, our fusion results excel in downstream tasks covering semantic segmentation and object detection, confirming the practical value in real-world adverse weather perception. The source code will be available at https://github.com/Feecuin/CAWM-Mamba.

Figures (9)

• Figure 1: The overall architecture of CAWM-Mamba. It consists of three main components: WAPM, CFIM, and WSSB. Within WSSB, Freq-SSM captures anisotropic high-frequency features, while CDSM learns a unified degradation representation for robust generalization.
• Figure 2: The distribution of degradation interference in the wavelet domain. The left image is a visible light image with degradation interference, while the right image shows the distribution in the wavelet domain.
• Figure 3: Freq-SSM scanning mechanism
• Figure 4: Comparison of fusion results of different methods in three scenarios with composite degradation interference.
• Figure 5: Comparison of fusion results of different methods in three scenarios with single degradation interference.