Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SpectraLift: Physics-Guided Spectral-Inversion Network for Self-Supervised Hyperspectral Image Super-Resolution

Ritik Shah, Marco F. Duarte

TL;DR

SpectraLift addresses hyperspectral super-resolution without requiring ground-truth HR-HSI or precise PSF calibration by casting fusion as per-pixel spectral inversion using the spectral response function (SRF). It trains a compact per-pixel MLP, the Spectral Inversion Network (SIN), to invert the SRF-based degradation with an $\ell_1$ loss on synthetic LR-MSI derived from LR-HSI, then applies the learned mapping to HR-MSI to produce HR-HSI. The approach yields blur- and resolution-agnostic performance, converges quickly, and delivers strong PSNR, SAM, SSIM, and RMSE across synthetic benchmarks, while remaining lightweight and interpretable. Real-world tests on UH data show competitive visual and spectral fidelity, despite unknown SRF and temporal misalignment, illustrating practical applicability with robust margins over unsupervised baselines and competitive results versus supervised methods.

Abstract

High-spatial-resolution hyperspectral images (HSI) are essential for applications such as remote sensing and medical imaging, yet HSI sensors inherently trade spatial detail for spectral richness. Fusing high-spatial-resolution multispectral images (HR-MSI) with low-spatial-resolution hyperspectral images (LR-HSI) is a promising route to recover fine spatial structures without sacrificing spectral fidelity. Most state-of-the-art methods for HSI-MSI fusion demand point spread function (PSF) calibration or ground truth high resolution HSI (HR-HSI), both of which are impractical to obtain in real world settings. We present SpectraLift, a fully self-supervised framework that fuses LR-HSI and HR-MSI inputs using only the MSI's Spectral Response Function (SRF). SpectraLift trains a lightweight per-pixel multi-layer perceptron (MLP) network using ($i$)~a synthetic low-spatial-resolution multispectral image (LR-MSI) obtained by applying the SRF to the LR-HSI as input, ($ii$)~the LR-HSI as the output, and ($iii$)~an $\ell_1$ spectral reconstruction loss between the estimated and true LR-HSI as the optimization objective. At inference, SpectraLift uses the trained network to map the HR-MSI pixel-wise into a HR-HSI estimate. SpectraLift converges in minutes, is agnostic to spatial blur and resolution, and outperforms state-of-the-art methods on PSNR, SAM, SSIM, and RMSE benchmarks.

SpectraLift: Physics-Guided Spectral-Inversion Network for Self-Supervised Hyperspectral Image Super-Resolution

TL;DR

SpectraLift addresses hyperspectral super-resolution without requiring ground-truth HR-HSI or precise PSF calibration by casting fusion as per-pixel spectral inversion using the spectral response function (SRF). It trains a compact per-pixel MLP, the Spectral Inversion Network (SIN), to invert the SRF-based degradation with an loss on synthetic LR-MSI derived from LR-HSI, then applies the learned mapping to HR-MSI to produce HR-HSI. The approach yields blur- and resolution-agnostic performance, converges quickly, and delivers strong PSNR, SAM, SSIM, and RMSE across synthetic benchmarks, while remaining lightweight and interpretable. Real-world tests on UH data show competitive visual and spectral fidelity, despite unknown SRF and temporal misalignment, illustrating practical applicability with robust margins over unsupervised baselines and competitive results versus supervised methods.

Abstract

High-spatial-resolution hyperspectral images (HSI) are essential for applications such as remote sensing and medical imaging, yet HSI sensors inherently trade spatial detail for spectral richness. Fusing high-spatial-resolution multispectral images (HR-MSI) with low-spatial-resolution hyperspectral images (LR-HSI) is a promising route to recover fine spatial structures without sacrificing spectral fidelity. Most state-of-the-art methods for HSI-MSI fusion demand point spread function (PSF) calibration or ground truth high resolution HSI (HR-HSI), both of which are impractical to obtain in real world settings. We present SpectraLift, a fully self-supervised framework that fuses LR-HSI and HR-MSI inputs using only the MSI's Spectral Response Function (SRF). SpectraLift trains a lightweight per-pixel multi-layer perceptron (MLP) network using ()~a synthetic low-spatial-resolution multispectral image (LR-MSI) obtained by applying the SRF to the LR-HSI as input, ()~the LR-HSI as the output, and ()~an spectral reconstruction loss between the estimated and true LR-HSI as the optimization objective. At inference, SpectraLift uses the trained network to map the HR-MSI pixel-wise into a HR-HSI estimate. SpectraLift converges in minutes, is agnostic to spatial blur and resolution, and outperforms state-of-the-art methods on PSNR, SAM, SSIM, and RMSE benchmarks.

Paper Structure

This paper contains 11 sections, 3 equations, 3 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Spectral Inversion Network (SIN)
  4. Rationale for Per-Pixel Formulation
  5. SIN Insights and Limitations
  6. Experiments and Analysis
  7. Quality Metrics
  8. Synthetic Datasets
  9. Real-World Data
  10. Ablation
  11. Conclusion

Figures (3)

  • Figure 1: The SpectraLift pipelines. Top: self-supervised training of the Spectral Inversion Network (SIN) via SRF-based spectral inversion. Bottom: pixel-wise inference on HR‐MSI with the trained SIN to produce the super resolved High-spatial-Resolution Hyperspectral Image (HR HSI)
  • Figure 2: Point Spread Functions used for Synthetic LR HSI generation
  • Figure 3: University of Houston super-resolved results and corresponding spectra for two test scenes. (a, c) Super-resolved images with zoomed-in crops. (b, d) Spectral plots for selected regions.