Content-driven Magnitude-Derivative Spectrum Complementary Learning for Hyperspectral Image Classification

TL;DR

This paper tackles spectral confusion in hyperspectral image (HSI) classification by exploiting complementary information from spectral magnitude and derivative spectra. It introduces the Content-driven Spectrum Complementary Network (CSCN), featuring Magnitude-Derivative Dual Encoders, a Content-adaptive Point-wise Fusion Module, and Hybrid Disparity-enhancing Loss to promote diverse, discriminative feature representations. Through extensive experiments on WHU-OHS and eight benchmark datasets, CSCN achieves state-of-the-art results and demonstrates robustness to spectral noise, with ablation studies confirming the effectiveness of each component and design choice. The approach highlights the value of per-pixel, content-aware fusion of magnitude and derivative information and points toward future integration with pre-trained spectral models such as SpectralGPT for scalable remote sensing analyses.

Abstract

Extracting discriminative information from complex spectral details in hyperspectral image (HSI) for HSI classification is pivotal. While current prevailing methods rely on spectral magnitude features, they could cause confusion in certain classes, resulting in misclassification and decreased accuracy. We find that the derivative spectrum proves more adept at capturing concealed information, thereby offering a distinct advantage in separating these confusion classes. Leveraging the complementarity between spectral magnitude and derivative features, we propose a Content-driven Spectrum Complementary Network based on Magnitude-Derivative Dual Encoder, employing these two features as combined inputs. To fully utilize their complementary information, we raise a Content-adaptive Point-wise Fusion Module, enabling adaptive fusion of dual-encoder features in a point-wise selective manner, contingent upon feature representation. To preserve a rich source of complementary information while extracting more distinguishable features, we introduce a Hybrid Disparity-enhancing Loss that enhances the differential expression of the features from the two branches and increases the inter-class distance. As a result, our method achieves state-of-the-art results on the extensive WHU-OHS dataset and eight other benchmark datasets.

Figures (12)

• Figure 1: (a) HSI cubes present the visualization of spectral magnitude and derivative features. In (b), spectral distributions of C2 and C3 are compared, showing strong similarity in magnitude distribution but significant differences in derivative. Subsequently, the classification confusion and accuracy decline in (c) of magnitude reflects its insufficient feature separation ability, where the accurate classification in derivative precisely compensates for it. Additionally, the visualization of the classification results (d) also confirms this characteristic.
• Figure 2: (a) Overall architecture of the proposed Content-driven Spectrum Complementary Network consists of the Magnitude-Derivative Dual Encoder, Content-adaptive Point-wise Fusion Module, and Hybrid Disparity-enhancing Loss. (b) The proposed fusion module receives features from the dual-encoder along with the fused features from the previous stage, refining and feeding them into the next stage through aggregation and refinement processes. (c) The proposed loss function integrates the rough predictions from the dual-encoder and receives features from the encoder and mini-decoder. This integration provides additional supervision to aid network learning. (In matrix visual results, darker colors indicate higher confidence in the classification results.)
• Figure 3: Comparison of Feature Maps from dual encoder as well as shallow and deep features. Deep features contain richer high-order semantic information, resulting in more explicit feature representation. It tends to be consistent in certain regions (first row) and exhibits notable differences in others (second row). To extract more complementary information, our goal is to enhance or preserve feature diversity between the two branches.
• Figure 4: Visualization results on WHU-OHS dataset. In cases where the similarity in magnitude spectral distribution leads to feature confusion, our method benefits from the assistance of the derivative spectrum, achieving more accurate classification.
• Figure 5: Visualization results on IP dataset. For a detailed inspection, we zoom in on a specific section of the map.