Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MambaMoE: Mixture-of-Spectral-Spatial-Experts State Space Model for Hyperspectral Image Classification

Yichu Xu, Di Wang, Hongzan Jiao, Lefei Zhang, Liangpei Zhang

TL;DR

The paper tackles hyperspectral image (HSI) classification by addressing directional heterogeneity and receptive-field limitations in patch-based Mamba models. It introduces MambaMoE, a spectral-spatial mixture-of-experts framework with Mixture of Mamba Expert Blocks (MoMEB) consisting of Spatial Routed Experts (SRE) and a Spectral Shared Expert (SSE) within a Dynamic Spectral-Spatial Expert Mamba (DSSEM), plus a lightweight routing module and an uncertainty-guided corrective learning (UGCL) decoder strategy. The approach employs image-level input, sparse expert activation, and dynamic supervision from uncertain regions to enhance feature learning in challenging areas, achieving state-of-the-art accuracy and efficiency on multiple public benchmarks. Ablation and visualization studies confirm the necessity of DSSEM components and UGCL, while showing interpretable directional expert usage aligned with land-cover geometry; the work demonstrates practical impact for accurate, efficient HSI classification with potential for real-time remote sensing applications.

Abstract

Mamba-based models have recently demonstrated significant potential in hyperspectral image (HSI) classification, primarily due to their ability to perform contextual modeling with linear computational complexity. However, existing Mamba-based approaches often overlook the directional modeling heterogeneity across different land-cover types, leading to limited classification performance. To address these limitations, we propose MambaMoE, a novel spectral-spatial Mixture-of-Experts (MoE) framework, which represents the first MoE-based approach in the HSI classification domain. Specifically, we design a Mixture of Mamba Expert Block (MoMEB) that performs adaptive spectral-spatial feature modeling via a sparse expert activation mechanism. Additionally, we introduce an uncertainty-guided corrective learning (UGCL) strategy that encourages the model to focus on complex regions prone to prediction ambiguity. This strategy dynamically samples supervision signals from regions with high predictive uncertainty, guiding the model to adaptively refine feature representations and thereby enhancing its focus on challenging areas. Extensive experiments conducted on multiple public HSI benchmark datasets show that MambaMoE achieves state-of-the-art performance in both classification accuracy and computational efficiency compared to existing advanced methods, particularly Mamba-based ones. The code will be available online at https://github.com/YichuXu/MambaMoE.

MambaMoE: Mixture-of-Spectral-Spatial-Experts State Space Model for Hyperspectral Image Classification

TL;DR

The paper tackles hyperspectral image (HSI) classification by addressing directional heterogeneity and receptive-field limitations in patch-based Mamba models. It introduces MambaMoE, a spectral-spatial mixture-of-experts framework with Mixture of Mamba Expert Blocks (MoMEB) consisting of Spatial Routed Experts (SRE) and a Spectral Shared Expert (SSE) within a Dynamic Spectral-Spatial Expert Mamba (DSSEM), plus a lightweight routing module and an uncertainty-guided corrective learning (UGCL) decoder strategy. The approach employs image-level input, sparse expert activation, and dynamic supervision from uncertain regions to enhance feature learning in challenging areas, achieving state-of-the-art accuracy and efficiency on multiple public benchmarks. Ablation and visualization studies confirm the necessity of DSSEM components and UGCL, while showing interpretable directional expert usage aligned with land-cover geometry; the work demonstrates practical impact for accurate, efficient HSI classification with potential for real-time remote sensing applications.

Abstract

Mamba-based models have recently demonstrated significant potential in hyperspectral image (HSI) classification, primarily due to their ability to perform contextual modeling with linear computational complexity. However, existing Mamba-based approaches often overlook the directional modeling heterogeneity across different land-cover types, leading to limited classification performance. To address these limitations, we propose MambaMoE, a novel spectral-spatial Mixture-of-Experts (MoE) framework, which represents the first MoE-based approach in the HSI classification domain. Specifically, we design a Mixture of Mamba Expert Block (MoMEB) that performs adaptive spectral-spatial feature modeling via a sparse expert activation mechanism. Additionally, we introduce an uncertainty-guided corrective learning (UGCL) strategy that encourages the model to focus on complex regions prone to prediction ambiguity. This strategy dynamically samples supervision signals from regions with high predictive uncertainty, guiding the model to adaptively refine feature representations and thereby enhancing its focus on challenging areas. Extensive experiments conducted on multiple public HSI benchmark datasets show that MambaMoE achieves state-of-the-art performance in both classification accuracy and computational efficiency compared to existing advanced methods, particularly Mamba-based ones. The code will be available online at https://github.com/YichuXu/MambaMoE.
Paper Structure (16 sections, 15 equations, 8 figures, 7 tables)

This paper contains 16 sections, 15 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Mamba-based HSI Classification
  4. Mixture-of-Experts
  5. Methodology
  6. Overview
  7. Mixture of Mamba Expert Block
  8. Uncertainty-Guided Corrective Learning
  9. Training Loss
  10. Experiment
  11. Dataset
  12. Experimental Settings
  13. Ablation Study
  14. Performance Comparison and Analysis
  15. Visualization
  16. ...and 1 more sections

Figures (8)

  • Figure 1: An illustration of the proposed MambaMoE. The core component, MoMEB incorporates DSSEM, which leverages SRE and SSE to dynamically capture spectral-spatial features. The Decoder is composed of FFBs and UARBs. After feature extraction and refinement, we adopt the fully connected layer as the classification head to generate the final predictions.
  • Figure 2: The overview of the UARB.
  • Figure 3: Visualization of the classification maps on Pavia University dataset. (a) False-color image. (b) Ground truth. (c) GAHT. (d) CSIL. (e) MASSFormer. (f) DSFormer. (g) SS-Mamba. (h) HyperMamba. (i) S2Mamba. (j) MambaLG. (k) MambaHSI. (l) MambaMoE.
  • Figure 4: Visualization of the classification maps on Houston dataset. (a) False-color image. (b) Ground truth. (c) GAHT. (d) CSIL. (e) MASSFormer. (f) DSFormer. (g) SS-Mamba. (h) HyperMamba. (i) S2Mamba. (j) MambaLG. (k) MambaHSI. (l) MambaMoE.
  • Figure 5: Visualization of the classification maps on Whu-HanChuan dataset. (a) False-color image. (b) Ground truth. (c) GAHT. (d) CSIL. (e) MASSFormer. (f) DSFormer. (g) SS-Mamba. (h) HyperMamba. (i) S2Mamba. (j) MambaLG. (k) MambaHSI. (l) MambaMoE.
  • ...and 3 more figures