HSRMamba: Efficient Wavelet Stripe State Space Model for Hyperspectral Image Super-Resolution

TL;DR

This work introduces a strip-based scanning scheme to effectively reduce artifacts from global unidirectional scanning and uses wavelet decomposition to alleviate modal conflicts between high-frequency spatial features and low-frequency spectral features, further improving super-resolution performance.

Abstract

Single hyperspectral image super-resolution (SHSR) aims to restore high-resolution images from low-resolution hyperspectral images. Recently, the Visual Mamba model has achieved an impressive balance between performance and computational efficiency. However, due to its 1D scanning paradigm, the model may suffer from potential artifacts during image generation. To address this issue, we propose HSRMamba. While maintaining the computational efficiency of Visual Mamba, we introduce a strip-based scanning scheme to effectively reduce artifacts from global unidirectional scanning. Additionally, HSRMamba uses wavelet decomposition to alleviate modal conflicts between high-frequency spatial features and low-frequency spectral features, further improving super-resolution performance. Extensive experiments show that HSRMamba not only excels in reducing computational load and model size but also outperforms existing methods, achieving state-of-the-art results.

Figures (6)

• Figure 1: (a) 1D global selective scanning MambaIR. (b) window selective scanning windowMambalocalmamba. (c) stripe 2D selective scanning in this work. Due to the presence of the cross-window, stripe scanning can more effectively capture both global and local features.
• Figure 2: The overall structure of HSRMamba. DWT represents Discrete Wavelet Transform, and IWT represents Inverse Discrete Wavelet Transform.
• Figure 3: The architecture of the Strip 2D selective scanning module (stripe length is 2). S6 block is the state space model in mamba.
• Figure 4: The structure of LFSE and HFSE. DSConv represents a $3 \times 3$ depth-wise separable convolution. VSSM represents the Visual State Space Model in MambaIR MambaIR with stripe scanning. CA represents the Squeeze-and-Excitation operation SEnet, while Head and Tail refer to standard convolutions used for channel dimension mapping, with a channel scaling factor of 8.
• Figure 5: Pseudo-color images (R=20, G=30, B=40) generated by all comparative models for the testing area of the PaviaU dataset for $\times4$ SR and their corresponding SAM error maps.