Enhancing Knowledge Transfer in Hyperspectral Image Classification via Cross-scene Knowledge Integration

TL;DR

The paper addresses cross-scene hyperspectral image classification under fully heterogeneous conditions, where spectral shifts from different sensors and non-overlapping label spaces hinder knowledge transfer. It introduces Cross-scene Knowledge Integration (CKI), a framework composed of Alignment of Spectral Characteristics (ASC), Cross-scene Knowledge Sharing Preference (CKSP), and Complementary Information Integration (CII) to align spectra, identify shared semantics, and incorporate target-private information. The approach combines a domain-agnostic projection, a sample-level semantic similarity mechanism, and a dual-teacher distillation scheme to produce a complete, robust target model. Experiments on Indian Pines, Pavia University, and Houston 2013 demonstrate state-of-the-art performance and strong stability across diverse cross-scene transfer scenarios, highlighting CKI's practical value for multi-sensor remote sensing tasks with limited target labels.

Abstract

Knowledge transfer has strong potential to improve hyperspectral image (HSI) classification, yet two inherent challenges fundamentally restrict effective cross-domain transfer: spectral variations caused by different sensors and semantic inconsistencies across heterogeneous scenes. Existing methods are limited by transfer settings that assume homogeneous domains or heterogeneous scenarios with only co-occurring categories. When label spaces do not overlap, they further rely on complete source-domain coverage and therefore overlook critical target-private information. To overcome these limitations and enable knowledge transfer in fully heterogeneous settings, we propose Cross-scene Knowledge Integration (CKI), a framework that explicitly incorporates target-private knowledge during transfer. CKI includes: (1) Alignment of Spectral Characteristics (ASC) to reduce spectral discrepancies through domain-agnostic projection; (2) Cross-scene Knowledge Sharing Preference (CKSP), which resolves semantic mismatch via a Source Similarity Mechanism (SSM); and (3) Complementary Information Integration (CII) to maximize the use of target-specific complementary cues. Extensive experiments verify that CKI achieves state-of-the-art performance with strong stability across diverse cross-scene HSI scenarios.

Figures (6)

• Figure 1: The general scenario studied in this paper, where source and target scenes are characterized by disparate spectral features attributable to distinct HSI sensors such as AVIRIS and CASI. Furthermore, there exists a mismatch in the label spaces between the label spaces of the source (encompassing categories including "Buildings-Grass-Trees-Drives", "Corn", and "Woods") and the target scenes ("Road", "Residential", "Commercial", "Parking Lot", "Grass", "Trees"). The intersection of these label spaces contains coarse-grained categories (e.g., "Buildings-Grass-Trees-Drives” and "Woods") and fine-grained categories (e.g., "Residential”, "Commercial", and "Grass”). In contrast, outlier classes—--particularly "Corn" from the source and "Parking Lot" from the target scenes---lie outside this overlap.
• Figure 2: Our method comprises three components: Alignment of Spectral Characteristics (ASC), Cross-scene Knowledge Sharing Preference (CKSP), and Complementary Information Integration (CII). ASC addresses spectral discrepancies caused by different HSI sensors by encoding the input data $x^s_{i}$ and $x^t_{j}$ into a domain-agnostic space using an adversarial training scheme with two transformation functions, $F_s$ and $F_t$. CKSP resolves semantic mismatches due to category correspondence issues by identifying shared semantics through an introduced Source Similarity Mechanism, ensuring adaptive transfer. CKSP includes an Interaction Fusion Spatial-Spectral (IFSS) Transformer learner and task heads $T_{s}$ and $T_{t}$. Finally, CII tackles the deficiency in utilizing target-private information within shared knowledge, caused by domain and semantic mismatches, by introducing complementary extraction and distillation integration to build the final integrated target student model with $G_{stu}$ and $T_{stu}$.
• Figure 3: ASC utilizes two transformation encoders, $F_s$ and $F_t$, to project the input data $x^s_{i}$ and $x^t_{j}$ into a domain-agnostic space via an adversarial discriminator $I$. The IFSS transformer network $G$ is shared by both the source and target data, while $T_s$ and $T_t$ serve as the classification heads for the source and target, respectively.
• Figure 4: CKSP leverages a Source Similarity Mechanism to determine the importance of each source sample during the knowledge transfer from source to target. This mechanism gives greater weight $\omega^s(x)$ to source samples with higher semantic similarity though entropy uncertainty $H(\widehat{y^s})$ and domain similarity ${I'}(F_s(x^s))$ to enhance knowledge sharing.
• Figure 5: CII contains Complementary Extraction (CE) and Distillation Integration (DI) modules to ensure the complete information of the target data. We employ Partial Distance Correlation $E_{DC}$ to find the target-private information to identify target-private information that supplements the shared information between the source and target. To further guarantee the completeness of the target information, we introduce a distillation integration method, with $E_{KL1}$ learning from shared information and $E_{KL2}$ learning from complementary information, thereby enhancing the representation of the entire target scene.