Spectral-Spatial Self-Supervised Learning for Few-Shot Hyperspectral Image Classification
Wenchen Chen, Yanmei Zhang, Zhongwei Xiao, Jianping Chu, Xingbo Wang
TL;DR
S4L-FSC addresses the scarcity of labeled data in hyperspectral image classification by decoupling spatial and spectral learning into staged pretraining on heterogeneous and homogeneous datasets, respectively. It introduces Rotation-Mirror SSL to enrich spatial representations and Masked Reconstruction SSL to inject spectral priors, both integrated with few-shot learning. A final target-domain fusion stage uses contrastive learning to leverage limited labeled samples, achieving superior performance across UP, IP, SA, and HC datasets with strong robustness to 1–5 labels per class. The approach yields tangible improvements in OA, AA, and Kappa while maintaining reasonable training efficiency, making it practical for cross-domain HSI tasks.
Abstract
Few-shot classification of hyperspectral images (HSI) faces the challenge of scarce labeled samples. Self-Supervised learning (SSL) and Few-Shot Learning (FSL) offer promising avenues to address this issue. However, existing methods often struggle to adapt to the spatial geometric diversity of HSIs and lack sufficient spectral prior knowledge. To tackle these challenges, we propose a method, Spectral-Spatial Self-Supervised Learning for Few-Shot Hyperspectral Image Classification (S4L-FSC), aimed at improving the performance of few-shot HSI classification. Specifically, we first leverage heterogeneous datasets to pretrain a spatial feature extractor using a designed Rotation-Mirror Self-Supervised Learning (RM-SSL) method, combined with FSL. This approach enables the model to learn the spatial geometric diversity of HSIs using rotation and mirroring labels as supervisory signals, while acquiring transferable spatial meta-knowledge through few-shot learning. Subsequently, homogeneous datasets are utilized to pretrain a spectral feature extractor via a combination of FSL and Masked Reconstruction Self-Supervised Learning (MR-SSL). The model learns to reconstruct original spectral information from randomly masked spectral vectors, inferring spectral dependencies. In parallel, FSL guides the model to extract pixel-level discriminative features, thereby embedding rich spectral priors into the model. This spectral-spatial pretraining method, along with the integration of knowledge from heterogeneous and homogeneous sources, significantly enhances model performance. Extensive experiments on four HSI datasets demonstrate the effectiveness and superiority of the proposed S4L-FSC approach for few-shot HSI classification.