Traffic Image Restoration under Adverse Weather via Frequency-Aware Mamba

TL;DR

The paper tackles restoring traffic images captured in adverse weather by integrating frequency-domain priors with a fast sequence model. It introduces Frequency-Aware Mamba (FAMamba) comprising the Dual-Branch Feature Extraction Block, Prior-Guided Block, and Adaptive Frequency Scanning Mechanism to fuse wavelet-domain priors with 2D Mamba scanning. Through extensive experiments on synthetic and real datasets, FA-Mamba outperforms state-of-the-art methods in PSNR, SSIM, and perceptual metrics while maintaining efficiency, and it enhances downstream tasks like object detection and segmentation. This approach offers a practical, scalable solution for robust traffic vision in challenging weather conditions.

Abstract

Traffic image restoration under adverse weather conditions remains a critical challenge for intelligent transportation systems. Existing methods primarily focus on spatial-domain modeling but neglect frequency-domain priors. Although the emerging Mamba architecture excels at long-range dependency modeling through patch-wise correlation analysis, its potential for frequency-domain feature extraction remains unexplored. To address this, we propose Frequency-Aware Mamba (FAMamba), a novel framework that integrates frequency guidance with sequence modeling for efficient image restoration. Our architecture consists of two key components: (1) a Dual-Branch Feature Extraction Block (DFEB) that enhances local-global interaction via bidirectional 2D frequency-adaptive scanning, dynamically adjusting traversal paths based on sub-band texture distributions; and (2) a Prior-Guided Block (PGB) that refines texture details through wavelet-based high-frequency residual learning, enabling high-quality image reconstruction with precise details. Meanwhile, we design a novel Adaptive Frequency Scanning Mechanism (AFSM) for the Mamba architecture, which enables the Mamba to achieve frequency-domain scanning across distinct subgraphs, thereby fully leveraging the texture distribution characteristics inherent in subgraph structures. Extensive experiments demonstrate the efficiency and effectiveness of FAMamba.

Figures (13)

• Figure 1: Comparison of different modeling methods. Our FA-Mamba can enhance the ability of the Mamba to restore image texture details from the frequency perspective.
• Figure 2: The overall architecture of the proposed Frequency-Aware Mamba (FA-Mamba). FA-Mamba mainly contains a series of Frequency-Aware Blocks (FA-Blocks) and High-Frequency Enhancement Module (HFEM). Specifically, the FA-Block is composed of a Dual-Branch Feature Extraction Block (DFEB) and a Prior-Guided Block (PGB).
• Figure 3: The comparison between (a) the 2D scanning strategy employed by VMambaliu2024vmamba and (b) our proposed Adaptive Frequency Scanning Mechanism (AFSM). Utilizing the wavelet Transform, we decompose the input into 4 frequency bands. It is then scanned along the frequency dimension in the spatial domain. Specifically, this strategy captures complex image details at different frequencies by relying on the texture distribution of subgraphs and applying a feature scanning mechanism.
• Figure 4: Image desnowing comparison on Snow100K-S, Snow100K-M, and Snow100K-L test sets (from top to bottom). (a) snow images; (b)-(d) images restored by DDMSN, TransWeather, and GridFormer, respectively; (e) images restored by our proposed FA-Mamba; (f) clean images. Please zoom in for the best view.
• Figure 5: Visual comparison on raindrop dataset. (a) raindrop images; (b)-(d) images restored by TransWeather, WeatherDiff, and MambaIR, respectively; (e) images restored by our proposed FA-Mamba; (f) clean images. Please zoom in for the best view.