SR$^{2}$-Net: A General Plug-and-Play Model for Spectral Refinement in Hyperspectral Image Super-Resolution

TL;DR

SR$^{2}$-Net addresses spectral fidelity in hyperspectral image super-resolution by introducing a lightweight, plug-and-play rectifier that can be attached to any SR backbone without architectural changes. It adopts an enhance-then-rectify strategy consisting of the Hierarchical Spectral-Spatial Synergy Attention (H-S$^{3}$A) for cross-band interaction and the Manifold Consistency Rectification (MCR) to align predictions with a physically plausible spectral manifold, guided by a degradation-consistency loss for data fidelity. Empirically, SR$^{2}$-Net yields consistent improvements in spectral fidelity and overall reconstruction quality across multiple backbones and scales, with negligible computational overhead and strong cross-domain generalization. The work demonstrates that decoupling spectral correction from SR backbones and enforcing physical spectral structure can significantly enhance HSIs without sacrificing spatial detail, offering practical benefits for downstream hyperspectral analysis.

Abstract

HSI-SR aims to enhance spatial resolution while preserving spectrally faithful and physically plausible characteristics. Recent methods have achieved great progress by leveraging spatial correlations to enhance spatial resolution. However, these methods often neglect spectral consistency across bands, leading to spurious oscillations and physically implausible artifacts. While spectral consistency can be addressed by designing the network architecture, it results in a loss of generality and flexibility. To address this issue, we propose a lightweight plug-and-play rectifier, physically priors Spectral Rectification Super-Resolution Network (SR$^{2}$-Net), which can be attached to a wide range of HSI-SR models without modifying their architectures. SR$^{2}$-Net follows an enhance-then-rectify pipeline consisting of (i) Hierarchical Spectral-Spatial Synergy Attention (H-S$^{3}$A) to reinforce cross-band interactions and (ii) Manifold Consistency Rectification (MCR) to constrain the reconstructed spectra to a compact, physically plausible spectral manifold. In addition, we introduce a degradation-consistency loss to enforce data fidelity by encouraging the degraded SR output to match the observed low resolution input. Extensive experiments on multiple benchmarks and diverse backbones demonstrate consistent improvements in spectral fidelity and overall reconstruction quality with negligible computational overhead. Our code will be released upon publication.

Figures (4)

• Figure 1: Motivation of SR$^{2}$-Net. (a) General SR models tend to introduce spectral distortion during reconstruction. (b) Spectral consistency is achieved via the plug-and-play SR$^{2}$-Net rectifier.
• Figure 2: The overview of SR$^{2}$-Net, a plug-and-play rectifier for hyperspectral image super-resolution. Given an LR input $\textit{I}_{LR}$, an SR model produces an initial reconstruction $\tilde{\textit{I}}_{SR}$, which is then refined by SR$^{2}$-Net to obtain the rectified output $\hat{\textit{I}}_{SR}$. (a) H-S$^{3}$A enhances cross-band interaction via hierarchical spectral grouping and adjacent spectra shuffle. (b) TSA models complementary dependencies along spectral/height/width views and fuses them to recalibrate features. (c) MCR projects features to a compact spectral manifold and iteratively aggregates corrections for physically plausible spectra.
• Figure 3: Qualitative comparisons on CAVE (top) and ARAD-1K (bottom) at the $\times4$ scale. Pseudo-RGB visualizations are rendered using spectral bands 25--15--5 as R--G--B, and the corresponding error maps are shown below each reconstruction. Red boxes highlight representative regions where SR$^{2}$-Net reduces reconstruction errors around edges and fine structures compared with the baselines.
• Figure 4: Spectral fidelity visualization on CAVE dataset at the $\times4$ scale. For two representative pixels (marked in the left pseudo-RGB images), we plot the ground-truth spectra (GT), the baseline backbone outputs, and the corresponding results after attaching SR$^{2}$-Net. While GT spectra may contain mild fluctuations, baseline reconstructions often exhibit abnormal cross-band deviations or spurious oscillations; SR$^{2}$-Net suppresses these non-physical behaviors and produces spectra that better follow the GT trend across different backbones.