S4DL: Shift-sensitive Spatial-Spectral Disentangling Learning for Hyperspectral Image Unsupervised Domain Adaptation

TL;DR

This work tackles cross-scene hyperspectral image unsupervised domain adaptation by addressing spectral-channel shifts that traditional alignment methods overlook. It introduces S4DL, a framework combining gradient-guided spatial-spectral disentangling (GSSD), a shift-sensitive adaptive monitor (SSAM), and a reversible feature extractor (RFE) to separate domain-invariant from domain-specific channel information while preserving low-level details. The three components work together under a loss L_total = L_cls + λ_1 L_ortho + λ_2 L_dom to improve transferability across diverse scenes, as demonstrated on Houston, HyRANK, and S-H datasets where S4DL achieves state-of-the-art OA and Kappa. The approach offers a practical impact by enhancing cross-scene HSI classification with robust, computation-efficient domain adaptation that explicitly leverages spectral information for disentangling domain shifts.

Abstract

Unsupervised domain adaptation techniques, extensively studied in hyperspectral image (HSI) classification, aim to use labeled source domain data and unlabeled target domain data to learn domain invariant features for cross-scene classification. Compared to natural images, numerous spectral bands of HSIs provide abundant semantic information, but they also increase the domain shift significantly. In most existing methods, both explicit alignment and implicit alignment simply align feature distribution, ignoring domain information in the spectrum. We noted that when the spectral channel between source and target domains is distinguished obviously, the transfer performance of these methods tends to deteriorate. Additionally, their performance fluctuates greatly owing to the varying domain shifts across various datasets. To address these problems, a novel shift-sensitive spatial-spectral disentangling learning (S4DL) approach is proposed. In S4DL, gradient-guided spatial-spectral decomposition is designed to separate domain-specific and domain-invariant representations by generating tailored masks under the guidance of the gradient from domain classification. A shift-sensitive adaptive monitor is defined to adjust the intensity of disentangling according to the magnitude of domain shift. Furthermore, a reversible neural network is constructed to retain domain information that lies in not only in semantic but also the shallow-level detailed information. Extensive experimental results on several cross-scene HSI datasets consistently verified that S4DL is better than the state-of-the-art UDA methods. Our source code will be available at https://github.com/xdu-jjgs/S4DL.

Figures (11)

• Figure 1: Channel Variance and Model Performance. It shows mean of standard deviations of spectral channels and corresponding classification accuracies of UDA methods on the HyRANK dataset, where the standard deviation was computed by the activation values of feature maps for source and target domain data. The bar graph delineates accuracy, while the line graph reflects the mean inter-domain standard deviation of the model's channels. The experiments were conducted on the HyRANK dataset Karantzalos2018.
• Figure 2: Framework of the proposed S4DL, including Reversible Feature Extractor, Gradient-guided Spatial-Spectral Decomposition and Shift-Sensitive Adaptive Monitor.
• Figure 3: Model architecture of Reversible Feature Extractor. Light blue color represents low-level features and dark blue color represents high-level features.
• Figure 4: The pseudocolor image and ground truth map of Houston dataset. (a) Pseudocolor image of Houston 2013. (b) Pseudocolor image of Houston 2018. (c) Ground-truth map of Houston 2013. (d) Ground-truth map of Houston 2018.
• Figure 5: The pseudocolor image and ground truth map of HyRANK dataset. (a) Pseudocolor image of Dioni. (b) Pseudocolor image of Loukia. (c) Ground-truth map of Dioni. (d) Ground-truth map of Loukia.