MambaHSI: Spatial-Spectral Mamba for Hyperspectral Image Classification

TL;DR

MambaHSI introduces a pure-SSM, image-level framework for hyperspectral image classification that achieves long-range spatial modeling with linear complexity. It combines a spatial Mamba block, a spectral Mamba block, and a spatial-spectral fusion module to capture fine-grained pixel-level spatial details alongside spectral continuity, enabling adaptive fusion of spatial and spectral cues. The approach delivers state-of-the-art performance on four diverse HSI datasets and demonstrates clear efficiency advantages over patch-based and transformer-based methods. This work highlights the potential of state-space models as robust backbones for efficient, high-precision HSI analysis in practical settings.

Abstract

Transformer has been extensively explored for hyperspectral image (HSI) classification. However, transformer poses challenges in terms of speed and memory usage because of its quadratic computational complexity. Recently, the Mamba model has emerged as a promising approach, which has strong long-distance modeling capabilities while maintaining a linear computational complexity. However, representing the HSI is challenging for the Mamba due to the requirement for an integrated spatial and spectral understanding. To remedy these drawbacks, we propose a novel HSI classification model based on a Mamba model, named MambaHSI, which can simultaneously model long-range interaction of the whole image and integrate spatial and spectral information in an adaptive manner. Specifically, we design a spatial Mamba block (SpaMB) to model the long-range interaction of the whole image at the pixel-level. Then, we propose a spectral Mamba block (SpeMB) to split the spectral vector into multiple groups, mine the relations across different spectral groups, and extract spectral features. Finally, we propose a spatial-spectral fusion module (SSFM) to adaptively integrate spatial and spectral features of a HSI. To our best knowledge, this is the first image-level HSI classification model based on the Mamba. We conduct extensive experiments on four diverse HSI datasets. The results demonstrate the effectiveness and superiority of the proposed model for HSI classification. This reveals the great potential of Mamba to be the next-generation backbone for HSI models. Codes are available at https://github.com/li-yapeng/MambaHSI .

Figures (12)

• Figure 1: Motivation illustration. The local characteristics of CNNs and the quadratic complexity of Transformers limit their ability to achieve fine-grained global modeling. In contrast, the proposed MambaHSI model can achieve pixel-level fine-grained spatial feature modeling with linear complexity. By incorporating spectral sequence information, MambaHSI enhances the extraction of spectral features.
• Figure 2: Overview of the proposed MambaHSI framework. The whole hyperspectral image is fed into the embedding layer to obtain pixel-level embeddings. Then these embeddings are taken as the inputs of the encoder to model the long-range dependencies and capture the discriminative features. Finally, the segmentation head classifies the features extracted by the encoder to obtain the final prediction. The encoder block contains three components: spatial Mamba block for capturing spatial features, spectral Mamba block for extracting spectral features, and spatial-spectral fusion module to fuse the spatial and spectral features.
• Figure 3: Pavia Univeristy data set. (a) False color image. (b) Ground truth. (c) Category and sample settings.
• Figure 4: Houston data set. (a) False color image. (b) Ground truth. (c) Category and sample settings.
• Figure 5: HanChuan data set. (a) False color image. (b) Ground truth. (c) Category and sample settings.