Equal is Not Always Fair: A New Perspective on Hyperspectral Representation Non-Uniformity

TL;DR

The paper addresses pervasive non-uniformity in hyperspectral image representations across spatial, spectral, and feature dimensions. It introduces FairHyp, a fairness-directed framework comprising RK4-SVA for spatial variability, S2FairConv for feature sparsity, and SCSS for spectral dependencies, integrated into restoration and classification pipelines. Through extensive experiments on denoising, inpainting, super-resolution, and classification, FairHyp delivers state-of-the-art results and demonstrates the complementary, non-trade-off contributions of each module. The work offers a principled perspective on balancing adaptability, efficiency, and fidelity in high-dimensional vision tasks and suggests broader applicability of modular fairness-directed design in non-uniform data domains.

Abstract

Hyperspectral image (HSI) representation is fundamentally challenged by pervasive non-uniformity, where spectral dependencies, spatial continuity, and feature efficiency exhibit complex and often conflicting behaviors. Most existing models rely on a unified processing paradigm that assumes homogeneity across dimensions, leading to suboptimal performance and biased representations. To address this, we propose FairHyp, a fairness-directed framework that explicitly disentangles and resolves the threefold non-uniformity through cooperative yet specialized modules. We introduce a Runge-Kutta-inspired spatial variability adapter to restore spatial coherence under resolution discrepancies, a multi-receptive field convolution module with sparse-aware refinement to enhance discriminative features while respecting inherent sparsity, and a spectral-context state space model that captures stable and long-range spectral dependencies via bidirectional Mamba scanning and statistical aggregation. Unlike one-size-fits-all solutions, FairHyp achieves dimension-specific adaptation while preserving global consistency and mutual reinforcement. This design is grounded in the view that non-uniformity arises from the intrinsic structure of HSI representations, rather than any particular task setting. To validate this, we apply FairHyp across four representative tasks including classification, denoising, super-resolution, and inpaintin, demonstrating its effectiveness in modeling a shared structural flaw. Extensive experiments show that FairHyp consistently outperforms state-of-the-art methods under varied imaging conditions. Our findings redefine fairness as a structural necessity in HSI modeling and offer a new paradigm for balancing adaptability, efficiency, and fidelity in high-dimensional vision tasks.

Figures (13)

• Figure 1: Illustration of the trilemma in hyperspectral image (HSI) representation and the motivation of FairHyp. The red dashed triangle denotes the trade-offs among Spatial Adaptability, Spectral Long-Range Modeling, and Feature Efficiency, which constrain existing methods. FairHyp (shown as the yellow circular path) resolves this conflict through a modular, fairness-directed design that achieves balanced and adaptive HSI representation.
• Figure 2: Integration of proposed modules into two representative HSI pipelines. RK4-SVA, S$^2$Fair Conv, and SCSS are inserted into restoration (left) and classification (right) networks to address spatial, feature, and spectral non-uniformities, respectively. RK4-SVA is detailed separately in Fig. \ref{['fig-rk4sva']}.
• Figure 3: Spatial variability across imaging sources. Different sources produce different pixel spacing, containing fundamentally different spatial content—ranging from mixed-object regions to fine-grained details—thereby affecting spatial correlation patterns.
• Figure 4: Visualization revealing the breakdown of adjacency-based correlation assumptions in HSIs. (a) Band correlation heatmap of the Indian Pines dataset. (b) and (c) Top-k related band average distances for Pavia University and Indian Pines, respectively. Background colors indicate the adequacy of different receptive fields, as detailed in the text.
• Figure 5: Schematic of the RK4-based Spatial Variability Adapter (RK4-SVA), which integrates iterative refinement with reference-guided gated modulation for adaptive restoration of spatial correlations. The 3D heatmaps illustrate the estimated spatial correlation patterns at different refinement stages.