Knowledge-Aware Mamba for Joint Change Detection and Classification from MODIS Times Series

TL;DR

This work tackles the challenge of detecting land-cover change from MODIS time series by addressing mixed pixels, spatiotemporal-spectral coupling, and background class heterogeneity. The authors introduce Knowledge-aware Mamba (KAMamba), combining a knowledge-guided transition loss (KAT-loss), a multi-task learning objective (pre-change/post-change classification and change detection), and a spatial-spectral-temporal Mamba backbone with sparse, deformable tokens (SSTMamba) to disentangle signals while maintaining efficiency. Key contributions include the knowledge-guided transition modeling, the joint constraint learning with differential fusion, and the SST-Mamba/SDMamba design for long-range, low-cost processing. Evaluations on Saskatchewan MODIS data show about 1.5–6% gains in average F1 for change detection and roughly 2% improvements in OA, AA, and Kappa for LULC classification, demonstrating improved ecological plausibility and robustness to class imbalance.

Abstract

Although change detection using MODIS time series is critical for environmental monitoring, it is a highly challenging task due to key MODIS difficulties, e.g., mixed pixels, spatial-spectral-temporal information coupling effect, and background class heterogeneity. This paper presents a novel knowledge-aware Mamba (KAMamba) for enhanced MODIS change detection, with the following contributions. First, to leverage knowledge regarding class transitions, we design a novel knowledge-driven transition-matrix-guided approach, leading to a knowledge-aware transition loss (KAT-loss) that can enhance detection accuracies. Second, to improve model constraints, a multi-task learning approach is designed, where three losses, i.e., pre-change classification loss (PreC-loss), post-change classification loss (PostC-loss), and change detection loss (Chg-loss) are used for improve model learning. Third, to disentangle information coupling in MODIS time series, novel spatial-spectral-temporal Mamba (SSTMamba) modules are designed. Last, to improve Mamba model efficiency and remove computational cost, a sparse and deformable Mamba (SDMamba) backbone is used in SSTMamba. On the MODIS time-series dataset for Saskatchewan, Canada, we evaluate the method on land-cover change detection and LULC classification; results show about 1.5-6% gains in average F1 for change detection over baselines, and about 2% improvements in OA, AA, and Kappa for LULC classification.

Figures (5)

• Figure 1: Illustration of the Knowledge-Aware Mamba approach (KAMamba). The transition matrix $\textit{T}$ is built based on the existing prior knowledge. The model outputs three vectors, e.g., $\mathcal{L}_1$, $\mathcal{L}_2$, $\mathcal{L}_3$, the classification probability of 2010 MODIS data, 2015 MODIS data, and the change probability. Take urban and cropland as an example, it is very unlikely that urban will be changed into cropland, with only 0.01 probability. If the change detection head outputs a possibility of 0.8 or higher, that means that the results of change detection are not consistent with the prior knowledge, leading to a high penalty for overestimated results, which is realized by the knowledge-aware transition loss (KAT-Loss). While in the existing methods, they do not include any connections or relationships to establish the correlation between different loss functions, they only use weight coefficients to balance the contribution of each loss, leading to unreasonable change directions.
• Figure 2: Overview of our proposed Knowledge-aware Mamba for MODIS time series change detection. It is a sequential model that contains the Spatial-Spectral-Temporal Sparse Module (SST-SM) without weight sharing and the Feature Differential Module (FDiffM). In SST-SM, Temporal Group Stem (TSG), Spatial Sparse Deformable Mamba (SpaSDM), Spectral Sparse Deformable Mamba (SpeSDM), and Temporal Sparse Deformable Mamba (TemSDM) are included; the corresponding output features are used for building the input feature of FDiffM. The differential features are concatenated and then fed to the change detection head.
• Figure 3: Overview of the Spatial-Spectral-Temporal Sparse Deformable Mamba.After TGS module, the feature is reshaped into $HW \times LT$ as spatial tokens and then fed into SDSpaM. Here, we set $L=12$. The output of SDSpaM is reshaped into $L \times T$ after doing the maxpooling on the spatial dimension as the input of SDSpeM. For SDTemM, the input is being the shape of $T \times HW$. All the spatial, spectral, and temporal modules use the attention mechanism to select dynamically the most informative tokens to reduce the redundancy.
• Figure 4: The transition matrix for building the knowledge-aware loss KAT-loss.
• Figure 5: Comparison of models for change detection and LULC map. The first row shows change detection results. The second and third rows present LULC classification results for 2010 and 2015, respectively.