Spectral Property-Driven Data Augmentation for Hyperspectral Single-Source Domain Generalization

Abstract

While hyperspectral images (HSI) benefit from numerous spectral channels that provide rich information for classification, the increased dimensionality and sensor variability make them more sensitive to distributional discrepancies across domains, which in turn can affect classification performance. To tackle this issue, hyperspectral single-source domain generalization (SDG) typically employs data augmentation to simulate potential domain shifts and enhance model robustness under the condition of single-source domain training data availability. However, blind augmentation may produce samples misaligned with real-world scenarios, while excessive emphasis on realism can suppress diversity, highlighting a tradeoff between realism and diversity that limits generalization to target domains. To address this challenge, we propose a spectral property-driven data augmentation (SPDDA) that explicitly accounts for the inherent properties of HSI, namely the device-dependent variation in the number of spectral channels and the mixing of adjacent channels. Specifically, SPDDA employs a spectral diversity module that resamples data from the source domain along the spectral dimension to generate samples with varying spectral channels, and constructs a channel-wise adaptive spectral mixer by modeling inter-channel similarity, thereby avoiding fixed augmentation patterns. To further enhance the realism of the augmented samples, we propose a spatial-spectral co-optimization mechanism, which jointly optimizes a spatial fidelity constraint and a spectral continuity self-constraint. Moreover, the weight of the spectral self-constraint is adaptively adjusted based on the spatial counterpart, thus preventing over-smoothing in the spectral dimension and preserving spatial structure. Extensive experiments conducted on three remote sensing benchmarks demonstrate that SPDDA outperforms state-of-the-art methods.

Figures (5)

• Figure 1: Overview of spectral property-driven data augmentation (SPDDA). Briefly, a spectral diversity module (SDM) is utilized to inject perturbations into source-domain samples $X_\mathrm{SD} \in \mathbb{R}^{H \times W \times C}$ to obtain extended-domain samples $X_\mathrm{ED}\in \mathbb{R}^{H \times W \times K}$, where $K$ is a random integer. SDM constructs a channel mask and a channel-wise adaptive spectral mixer to masks an arbitrary number of channels and mixes the remaining channels, which adheres to the practical phenomenon in hyperspectral image. The augmentation procedure is supervised by a spatial-spectral co-optimization mechanism, which is comprised of a spatial fidelity constraint and a spectral continuity self-constraint.
• Figure 2: Illustration of the variation in the spectral continuity self-constraint weight $\lambda$.
• Figure 3: Illustration of the pseudo-color images for the extended-domain scene for each method. Regions marked by the blue rectangle are enlarged and displayed in the lower right corner of each image.
• Figure 4: Comparison of realism and diversity. PSNR measures spatial realism, while the mean and standard deviation (Std) of SAM represent spectral realism and diversity, respectively. A lower mean SAM indicates higher spectral realism, and a greater Std means higher diversity.
• Figure 5: Effects of CASM, $\mathcal{L}_\mathrm{SF}$, and $\mathcal{L}_\mathrm{SC}$ on spectral angle mapper (SAM). The average SAM quantifies realism of generation. A lower average SAM indicates higher realism.