SSUMamba: Spatial-Spectral Selective State Space Model for Hyperspectral Image Denoising

TL;DR

SSUMamba tackles hyperspectral image denoising by modeling long-range spatial-spectral dependencies with a memory-efficient state-space framework. It introduces the spatial-spectral continuous scan (SSCS) Mamba block, which uses six bidirectional scan schemes to capture joint spatial-spectral correlations, augmented by residual 3D convolutions to exploit local structure. The architecture achieves state-of-the-art denoising on both synthetic and real HSIs while consuming less memory and enabling faster inference than transformer-based methods. This work demonstrates that carefully designed, continuous scanning within a Mamba-based SSM effectively leverages the 3D nature of HSIs, offering a practical, scalable solution for HSI restoration and potentially broader tasks like super-resolution and classification.

Abstract

Denoising is a crucial preprocessing step for hyperspectral images (HSIs) due to noise arising from intra-imaging mechanisms and environmental factors. Long-range spatial-spectral correlation modeling is beneficial for HSI denoising but often comes with high computational complexity. Based on the state space model (SSM), Mamba is known for its remarkable long-range dependency modeling capabilities and computational efficiency. Building on this, we introduce a memory-efficient spatial-spectral UMamba (SSUMamba) for HSI denoising, with the spatial-spectral continuous scan (SSCS) Mamba being the core component. SSCS Mamba alternates the row, column, and band in six different orders to generate the sequence and uses the bidirectional SSM to exploit long-range spatial-spectral dependencies. In each order, the images are rearranged between adjacent scans to ensure spatial-spectral continuity. Additionally, 3D convolutions are embedded into the SSCS Mamba to enhance local spatial-spectral modeling. Experiments demonstrate that SSUMamba achieves superior denoising results with lower memory consumption per batch compared to transformer-based methods. The source code is available at https://github.com/lronkitty/SSUMamba.

Figures (13)

• Figure 1: Comparison of our SSUMamba and transformer-based TRQ3D, SST, and SERT on the ICVL testing set. The results indicate that SSUMamba outperforms transformer-based methods in peak signal-to-noise ratio (PSNR) while utilizing fewer parameters and memory consumption per batch.
• Figure 2: SSUMamba uses the encoder-decoder framework, incorporating a feature extractor, encoder blocks, SSCS Mamba blocks, decoder blocks with skip connections, and a reconstructor.
• Figure 3: The original Mamba block and the introduced SSCS Mamba block. SSCS Mamba block contains residual blocks and a bidirectional SSM layer with SSCS.
• Figure 4: Illustration of the dependencies in Mamba and spatial-spectral dependencies in HSIs. The red arrows indicate the dependencies propagated.
• Figure 5: Illustration of our SSCS scheme. (a) The sweep scan. (b) A typical example of SSCS. (c) SSCS with six schemes.