Dual-Branch Residual Network for Cross-Domain Few-Shot Hyperspectral Image Classification with Refined Prototype

TL;DR

This work tackles cross-domain few-shot hyperspectral image classification, where domain shifts from sensors and environments hinder generalization. It introduces a dual-branch residual network that separately learns spatial and spectral features, fuses them into robust representations, and refines class prototypes through a Query‑Prototype contrastive loss; simultaneous domain alignment is achieved with maximum mean discrepancy. The approach is augmented with a dimension-matching mapping layer and an episodic training scheme, and its effectiveness is demonstrated on four public HSI datasets, where it outperforms several strong baselines. The results indicate improved intra-class compactness and inter-class separation of refined prototypes, along with stable cross-domain adaptation, highlighting the method's potential for practical, scalable HSI analysis across disparate sensing conditions.

Abstract

Convolutional neural networks (CNNs) are effective for hyperspectral image (HSI) classification, but their 3D convolutional structures introduce high computational costs and limited generalization in few-shot scenarios. Domain shifts caused by sensor differences and environmental variations further hinder cross-dataset adaptability. Metric-based few-shot learning (FSL) prototype networks mitigate this problem, yet their performance is sensitive to prototype quality, especially with limited samples. To overcome these challenges, a dual-branch residual network that integrates spatial and spectral features via parallel branches is proposed in this letter. Additionally, more robust refined prototypes are obtained through a regulation term. Furthermore, a kernel probability matching strategy aligns source and target domain features, alleviating domain shift. Experiments on four publicly available HSI datasets illustrate that the proposal achieves superior performance compared to other methods.

Figures (3)

• Figure 1: Architecture of the proposed method, which includes four main modules. First, the mapping layers are employed to unify the dimensionality of $D^S$ and $D^T$ after the data pre-processing. Second, a dual-branch feature extractor captures spatial and spectral features independently, followed by feature fusion for enhanced representation. $F_\psi(\cdot)$ represents the feature extractor with parameters $\psi$. Third, to obtain more robust refined prototypes, we use the Query-Prototype contrastive refinement Loss (QPL) to improve inter-class separability and minimize intra-class variation. Finally, the domain alignment module leverages maximum mean discrepancy (MMD) to align feature distributions, reducing domain shifts in HSI. Training alternates between source and target domains. When a training episode is completed on either source domain or target domain. The source domain total loss $L^s_{total}$(the target domain total loss $L^t_{total}$) will be back propagated to update the feature extractor parameters.
• Figure 2: Overview of the feature extraction module: A dual-branch network that extracts spatial and spectral features separately. The spatial branch uses asymmetric convolutions to capture spatial correlations, while the spectral branch employs layered convolutions to learn spectral patterns. Finally, the spatial and spectral features are fused, producing a 120-dimensional vector for the FSL stage.
• Figure 3: Classification result of different methods on the four data sets with different number of labeled samples. (a) IP. (b) SA.