Importance of Disjoint Sampling in Conventional and Transformer Models for Hyperspectral Image Classification

TL;DR

By separating training, validation, and test data without overlap, the proposed method facilitates a fairer evaluation of how well a model can classify pixels it was not exposed to during training or validation, and significantly improves a model's generalization.

Abstract

Disjoint sampling is critical for rigorous and unbiased evaluation of state-of-the-art (SOTA) models. When training, validation, and test sets overlap or share data, it introduces a bias that inflates performance metrics and prevents accurate assessment of a model's true ability to generalize to new examples. This paper presents an innovative disjoint sampling approach for training SOTA models on Hyperspectral image classification (HSIC) tasks. By separating training, validation, and test data without overlap, the proposed method facilitates a fairer evaluation of how well a model can classify pixels it was not exposed to during training or validation. Experiments demonstrate the approach significantly improves a model's generalization compared to alternatives that include training and validation data in test data. By eliminating data leakage between sets, disjoint sampling provides reliable metrics for benchmarking progress in HSIC. Researchers can have confidence that reported performance truly reflects a model's capabilities for classifying new scenes, not just memorized pixels. This rigorous methodology is critical for advancing SOTA models and their real-world application to large-scale land mapping with Hyperspectral sensors. The source code is available at https://github.com/mahmad00/Disjoint-Sampling-for-Hyperspectral-Image-Classification.

Figures (12)

• Figure 1: The HSI cube is initially divided into overlapping 3D patches, as described in Algorithm 1. Each patch is centered at a spatial point and covers a $WS \times WS$ pixel extent across all spectral bands. These patches are then used in Algorithm 2 to create a disjoint train, validation, and test splits based on the geographical locations of the HSI samples. The selected samples are inputted into various models for feature learning and optimization. The processed features are subsequently passed through a fully connected layer for classification, and the softmax function is applied to generate class probability distributions. These distributions are used to generate the final ground truth maps for the disjoint validation, disjoint test, and full HSI test sets.
• Figure 2: University of Houston Dataset: Geographical locations of the disjoint train, validation, and test samples presented in Table \ref{['Tab2']}.
• Figure 3: Indian Pines: Geographical locations of the disjoint train, validation, and test samples presented in Table \ref{['Tab1']}.
• Figure 4: Pavia University Dataset: Geographical locations of the disjoint train, validation, and test samples presented in Table \ref{['Tab3']}.
• Figure 5: Salinas Dataset: Geographical locations of the disjoint train, validation, and test samples presented in Table \ref{['Tab4']}.