Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning

Yingkai Zhang, Tao Zhang, Jing Nie, Ying Fu

TL;DR

An unmixing-based fusion framework that decouples spatial-spectral information to simultaneously mitigate the impact of unregistered fusion and enhance the learnability of SR models is proposed.

Abstract

Unregistered hyperspectral image (HSI) super-resolution (SR) typically aims to enhance a low-resolution HSI using an unregistered high-resolution reference image. In this paper, we propose an unmixing-based fusion framework that decouples spatial-spectral information to simultaneously mitigate the impact of unregistered fusion and enhance the learnability of SR models. Specifically, we first utilize singular value decomposition for initial spectral unmixing, preserving the original endmembers while dedicating the subsequent network to enhancing the initial abundance map. To leverage the spatial texture of the unregistered reference, we introduce a coarse-to-fine deformable aggregation module, which first estimates a pixel-level flow and a similarity map using a coarse pyramid predictor. It further performs fine sub-pixel refinement to achieve deformable aggregation of the reference features. The aggregative features are then refined via a series of spatial-channel abundance cross-attention blocks. Furthermore, a spatial-channel modulated fusion module is presented to merge encoder-decoder features using dynamic gating weights, yielding a high-quality, high-resolution HSI. Experimental results on simulated and real datasets confirm that our proposed method achieves state-of-the-art super-resolution performance. The code will be available at https://github.com/yingkai-zhang/UAFL.

Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning

TL;DR

An unmixing-based fusion framework that decouples spatial-spectral information to simultaneously mitigate the impact of unregistered fusion and enhance the learnability of SR models is proposed.

Abstract

Unregistered hyperspectral image (HSI) super-resolution (SR) typically aims to enhance a low-resolution HSI using an unregistered high-resolution reference image. In this paper, we propose an unmixing-based fusion framework that decouples spatial-spectral information to simultaneously mitigate the impact of unregistered fusion and enhance the learnability of SR models. Specifically, we first utilize singular value decomposition for initial spectral unmixing, preserving the original endmembers while dedicating the subsequent network to enhancing the initial abundance map. To leverage the spatial texture of the unregistered reference, we introduce a coarse-to-fine deformable aggregation module, which first estimates a pixel-level flow and a similarity map using a coarse pyramid predictor. It further performs fine sub-pixel refinement to achieve deformable aggregation of the reference features. The aggregative features are then refined via a series of spatial-channel abundance cross-attention blocks. Furthermore, a spatial-channel modulated fusion module is presented to merge encoder-decoder features using dynamic gating weights, yielding a high-quality, high-resolution HSI. Experimental results on simulated and real datasets confirm that our proposed method achieves state-of-the-art super-resolution performance. The code will be available at https://github.com/yingkai-zhang/UAFL.
Paper Structure (14 sections, 17 equations, 6 figures, 5 tables)

This paper contains 14 sections, 17 equations, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Registered HSI Super-Resolution
  4. Unregistered HSI Super-Resolution
  5. Unmixing-based Fusion Framework
  6. Motivation and Overall Framework
  7. Coarse-to-Fine Deformable Aggregation
  8. Spatial-Channel Abundance Cross-Attention
  9. Spatial-Channel Modulated Fusion
  10. Experiments
  11. Experimental Settings
  12. Main Results
  13. Ablation Study
  14. Conclusion

Figures (6)

  • Figure 1: Comparison of performance (PSNR), computational cost (FLOPs), and memory cost (Params). The vertical axis represents PSNR (dB), while the horizontal axis indicates FLOPs. The size of each bubble corresponds to the number of parameters.
  • Figure 2: The motivation. (c) The explicit alignment within the SSCH zhang2025unaligned is prone to introducing spatial distortions and textural artifacts into the warped image. (d) The result validates the principle that leveraging a well-aligned, high-fidelity abundance enables the reconstruction of the HR HSI from its LR counterpart.
  • Figure 3: The overall architecture. We adopt a multi-scale encoder-decoder based on unmixing with several key modules: (a) Coarse-to-Fine Deformable Aggregation module, (b) Spatial-Channel Abundance Cross-Attention module, and (c) Spatial-Channel Modulation Fusion module to super-resolve high-quality HR HSI.
  • Figure 4: Visual comparison on the simulated dataset, ICVL arad2016sparse. The ground truth and input HSIs are shown with band 20. The results of different methods under the scale factor ($\times4$) on the ICVL dataset with the error maps.
  • Figure 5: Visual comparison on the real dataset, REAL lai2024hyperspectral. The ground truth and input HSIs are shown with band 20. The results of different methods under the scale factor ($\times4$, $\times8$) on the REAL dataset with the error maps.
  • ...and 1 more figures