Open-Set Domain Generalization through Spectral-Spatial Uncertainty Disentanglement for Hyperspectral Image Classification

TL;DR

This work tackles open-set domain generalization for hyperspectral image classification by introducing Spectral-Spatial Uncertainty Disentanglement (SSUD). The framework combines Spectrum-Invariant Frequency Disentanglement (SIFD), Dual-Channel Residual Networks (DCRN), and Evidential Deep Learning (EDL) to estimate pathway-specific uncertainty and adaptively select the most reliable spectral, spatial, or combined features for each sample. Threshold calibration via synthetic unknown generation enables robust unknown rejection without access to target-domain data during training, and experiments across three cross-scene HSI tasks show competitive performance against domain adaptation methods while maintaining strong unknown-class rejection. The approach provides interpretable uncertainty-informed decisions and highlights a practical route toward open-set robustness in real-world hyperspectral applications, with future work focusing on class-specific thresholds and multi-source DG extensions.

Abstract

Open-set domain generalization (OSDG) tackles the dual challenge of recognizing unknown classes while simultaneously striving to generalize across unseen domains without using target data during training. In this article, an OSDG framework for hyperspectral image classification is proposed, centered on a new Spectral-Spatial Uncertainty Disentanglement mechanism. It has been designed to address the domain shift influencing both spectral, spatial and combined feature extraction pathways using evidential deep learning, after which the most reliable pathway for each sample is adaptively selected. The proposed framework is further integrated with frequency-domain feature extraction for domain-invariant representation learning, dual-channel residual networks for spectral-spatial feature extraction, and evidential deep learning based uncertainty quantification. Experiments conducted on three cross scene hyperspectral datasets, show that performance comparable to state-of-the-art domain adaptation methods can be achieved despite no access to target data, while high unknown-class rejection and known-class accuracy levels are maintained. The implementation will be available at github.com/amir-khb/UGOSDG upon acceptance.

Figures (15)

• Figure 1: Illustration of the proposed OSDG framework with a HSI dataset of known $K$ classes. The architecture consists of four main components: SIFD for domain-invariant feature extraction, DCRN for spectral-spatial feature learning, EDL for uncertainty quantification, SSUD for uncertainty disentanglement, and finally threshold calibration before the open-set classification. The domain-invariant features from SIFD are combined with the original input via element-wise addition before being fed into DCRN. The loss functions $\mathcal{L}_{domain}$ (domain invariance loss), $\mathcal{L}_{recon}$ (reconstruction loss), $\mathcal{L}_{edl}$ (evidential deep learning loss), and $\mathcal{L}_{cls}$ (classification loss) are computed at their respective modules and combined for end-to-end training.
• Figure 2: Outline of the SIFD component he2024decoupled.
• Figure 3: DCRN processes hyperspectral patches through parallel spectral (2D convolutions) and spatial (ResNet50-based) pathways, fusing features via element-wise addition. The numbers in the blocks indicate the spatial and channel dimensions (height$\times$width$\times$channels) of feature maps throughout the convolutional layers.
• Figure 4: PU–PC task. (a) Pseudo-color image of PU with 39,332 labeled pixels. (b) Ground-truth image of PU. (c) Pseudo-color image of PC with 82,181 labeled pixels. (d) Ground-truth image of PC. Class distribution (PU-PC): Background, Brick (3682-2685), Meadow (18649-3090), Tree (3064-7598), Bitumen (1330-7287), Bare Soil (5029-6584), Asphalt (6631-9248), Shadow (947-2863), Unknown (0-42826)
• Figure 5: HU13–HU18 task. (a) Pseudo-color image of HU13 with 9,176 labeled pixels. (b) Ground-truth image of HU13. (c) Pseudo-color image of HU18 with 504,644 labeled pixels. (d) Ground-truth image of HU18. Class distribution (HU13-HU18): Background, Grass Healthy (1449-9799), Grass Stressed (1444-32502), Trees (1432-18576), Water (507-266), Residential buildings (1464-39794), Non-residential buildings (1435-223789), Road (1445-45793), Unknown (0-134125)