Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HSSDCT: Factorized Spatial-Spectral Correlation for Hyperspectral Image Fusion

Chia-Ming Lee, Yu-Hao Ho, Yu-Fan Lin, Jen-Wei Lee, Li-Wei Kang, Chih-Chung Hsu

TL;DR

The paper tackles hyperspectral image fusion by reconciling the need for large receptive fields with computational efficiency. It introduces HSSDCT, which couples a Hierarchical Dense-Residue Transformer Block (HDRTB) with a Spatial-Spectral Correlation Layer (SSCL) to decouple and efficiently model spatial and spectral dependencies, reducing self-attention complexity from quadratic to linear. The dual-branch architecture processes LR-HSI spectrally and HR-MSI spatially, with HDRTBs expanding context and SSCL delivering factorized, low-cost fusion. Experiments on AVIRIS data demonstrate state-of-the-art fusion accuracy while achieving lower runtime and memory, highlighting the approach’s practicality for real-world remote sensing tasks. Overall, HSSDCT offers a scalable, efficient solution for high-fidelity HR-HSI reconstruction that balances accuracy and efficiency.

Abstract

Hyperspectral image (HSI) fusion aims to reconstruct a high-resolution HSI (HR-HSI) by combining the rich spectral information of a low-resolution HSI (LR-HSI) with the fine spatial details of a high-resolution multispectral image (HR-MSI). Although recent deep learning methods have achieved notable progress, they still suffer from limited receptive fields, redundant spectral bands, and the quadratic complexity of self-attention, which restrict both efficiency and robustness. To overcome these challenges, we propose the Hierarchical Spatial-Spectral Dense Correlation Network (HSSDCT). The framework introduces two key modules: (i) a Hierarchical Dense-Residue Transformer Block (HDRTB) that progressively enlarges windows and employs dense-residue connections for multi-scale feature aggregation, and (ii) a Spatial-Spectral Correlation Layer (SSCL) that explicitly factorizes spatial and spectral dependencies, reducing self-attention to linear complexity while mitigating spectral redundancy. Extensive experiments on benchmark datasets demonstrate that HSSDCT delivers superior reconstruction quality with significantly lower computational costs, achieving new state-of-the-art performance in HSI fusion. Our code is available at https://github.com/jemmyleee/HSSDCT.

HSSDCT: Factorized Spatial-Spectral Correlation for Hyperspectral Image Fusion

TL;DR

The paper tackles hyperspectral image fusion by reconciling the need for large receptive fields with computational efficiency. It introduces HSSDCT, which couples a Hierarchical Dense-Residue Transformer Block (HDRTB) with a Spatial-Spectral Correlation Layer (SSCL) to decouple and efficiently model spatial and spectral dependencies, reducing self-attention complexity from quadratic to linear. The dual-branch architecture processes LR-HSI spectrally and HR-MSI spatially, with HDRTBs expanding context and SSCL delivering factorized, low-cost fusion. Experiments on AVIRIS data demonstrate state-of-the-art fusion accuracy while achieving lower runtime and memory, highlighting the approach’s practicality for real-world remote sensing tasks. Overall, HSSDCT offers a scalable, efficient solution for high-fidelity HR-HSI reconstruction that balances accuracy and efficiency.

Abstract

Hyperspectral image (HSI) fusion aims to reconstruct a high-resolution HSI (HR-HSI) by combining the rich spectral information of a low-resolution HSI (LR-HSI) with the fine spatial details of a high-resolution multispectral image (HR-MSI). Although recent deep learning methods have achieved notable progress, they still suffer from limited receptive fields, redundant spectral bands, and the quadratic complexity of self-attention, which restrict both efficiency and robustness. To overcome these challenges, we propose the Hierarchical Spatial-Spectral Dense Correlation Network (HSSDCT). The framework introduces two key modules: (i) a Hierarchical Dense-Residue Transformer Block (HDRTB) that progressively enlarges windows and employs dense-residue connections for multi-scale feature aggregation, and (ii) a Spatial-Spectral Correlation Layer (SSCL) that explicitly factorizes spatial and spectral dependencies, reducing self-attention to linear complexity while mitigating spectral redundancy. Extensive experiments on benchmark datasets demonstrate that HSSDCT delivers superior reconstruction quality with significantly lower computational costs, achieving new state-of-the-art performance in HSI fusion. Our code is available at https://github.com/jemmyleee/HSSDCT.
Paper Structure (9 sections, 5 equations, 5 figures, 1 table)

This paper contains 9 sections, 5 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. PROPOSED METHOD
  3. Overall Framework
  4. Hierarchical Dense-Residue Transformer Block
  5. Spatial-Spectral Correlation Layer
  6. Experimental Results
  7. Experiment Settings
  8. Quantitative Results
  9. Conclusion

Figures (5)

  • Figure 1: Overview of the proposed HSSDCT framework. The network takes an LR-HSI and an HR-MSI as inputs. The outputs of two parallel branches are finally fused to reconstruct the HR-HSI.
  • Figure 2: HSSDCT integrates the HDRTB for enlarged-receptive-field feature extraction using progressively enlarged windows, and the SSCL for efficient factorized spatial-spectral correlation modeling with linear complexity.
  • Figure 3: Hierarchical Dense-Residue Transformer Block (HDRTB) with hierarchical windows and dense connections, effectively enlarging the receptive field of features.
  • Figure 4: Spatial-Spectral Correlation Layer (SSCL) factorizes spatial and spectral correlations for efficient aggregation, reducing quadratic attention cost while preserving both spatial context and spectral fidelity.
  • Figure 5: Visualization of fused results and residue images for LR-HSI and HR-MSI inputs, comparing HSSDCT with representative baselines.