A Universal Knowledge Embedded Contrastive Learning Framework for Hyperspectral Image Classification

TL;DR

The paper tackles the gap between laboratory-style HSI models and real-world deployment by addressing unrealistic data splits and label scarcity. It introduces KnowCL, a universal knowledge-embedded contrastive learning framework that unifies supervised, unsupervised, and semi-supervised HSI classification through a flexible data-processing pipeline and an adaptive loss fusion strategy. The approach supports diverse backbones, including ViT-based and ResNet-based networks, and demonstrates strong performance across four benchmark datasets with disjoint sampling, often outperforming state-of-the-art baselines. The contributions offer a scalable path toward robust, data-efficient HSI classification applicable to practical remote sensing tasks, with code available online.

Abstract

Hyperspectral image (HSI) classification techniques have been intensively studied and a variety of models have been developed. However, these HSI classification models are confined to pocket models and unrealistic ways of dataset partitioning. The former limits the generalization performance of the model and the latter is partitioned leading to inflated model evaluation metrics, which results in plummeting model performance in the real world. Therefore, we propose a universal knowledge embedded contrastive learning framework (KnowCL) for supervised, unsupervised, and semisupervised HSI classification, which largely closes the gap between HSI classification models between pocket models and standard vision backbones. We present a new HSI processing pipeline in conjunction with a range of data transformation and augmentation techniques that provide diverse data representations and realistic data partitioning. The proposed framework based on this pipeline is compatible with all kinds of backbones and can fully exploit labeled and unlabeled samples with the expected training time. Furthermore, we design a new loss function, which can adaptively fuse the supervised loss and unsupervised loss, enhancing the learning performance. This proposed new classification paradigm shows great potential in exploring for HSI classification technology. The code can be accessed at \url{https://github.com/quanweiliu/KnowCL}.

Figures (12)

• Figure 1: Performance of KnowCL on DFC2018 using ViT and ResNet architecture, compared to other unsupervised and supervised baselines.
• Figure 2: The KnowCL framework for HSI classification. The framework is fed with the labeled samples patch $x$, and unlabeled samples patch $\hat{x}$ in supervised training and unsupervised training, respectively. For semi-supervised training, $x$ and $\hat{x}$ are fed jointly into the network.
• Figure 3: Data processing overview.
• Figure 4: Detailed structures of network components: (a) the attention block, (b) the supervised head, (c) the contrastive head.
• Figure 5: The training and test ground-truth maps on the four reference datasets, i.e., (a) UP, (b) Dioni, (c) Salinas, (d) DFC2018.