Exploiting Frequency Correlation for Hyperspectral Image Reconstruction

TL;DR

This work tackles hyperspectral image reconstruction from compressed measurements by introducing a Hyperspectral Frequency Correlation (HFC) prior inferred from frequency-domain statistics of HSIs. It develops two frequency-domain blocks, SAF and SIF, with a learnable gate and couples them with space-domain transformers in a Correlation-driven Mixing Domains Transformer (CMDT) within a deep frequency unfolding framework. Empirical results on CAVE/KAIST demonstrate state-of-the-art reconstruction quality and efficiency, with strong robustness to noise and extensive ablations validating component contributions. The approach highlights the value of learning in the frequency domain for HSIs and points toward broad applicability in spectral super-resolution and related analysis tasks.

Abstract

Deep priors have emerged as potent methods in hyperspectral image (HSI) reconstruction. While most methods emphasize space-domain learning using image space priors like non-local similarity, frequency-domain learning using image frequency priors remains neglected, limiting the reconstruction capability of networks. In this paper, we first propose a Hyperspectral Frequency Correlation (HFC) prior rooted in in-depth statistical frequency analyses of existent HSI datasets. Leveraging the HFC prior, we subsequently establish the frequency domain learning composed of a Spectral-wise self-Attention of Frequency (SAF) and a Spectral-spatial Interaction of Frequency (SIF) targeting low-frequency and high-frequency components, respectively. The outputs of SAF and SIF are adaptively merged by a learnable gating filter, thus achieving a thorough exploitation of image frequency priors. Integrating the frequency domain learning and the existing space domain learning, we finally develop the Correlation-driven Mixing Domains Transformer (CMDT) for HSI reconstruction. Extensive experiments highlight that our method surpasses various state-of-the-art (SOTA) methods in reconstruction quality and computational efficiency.

Figures (13)

• Figure 1: PSNR-Params-FLOPs comparisons of our method and SOTA methods. The scale of circles matches with parameters (M).
• Figure 2: The first row shows the gray images of Scene 9 in ascending spectral bands. The second row exhibits the spectrograms in ascending spectral bands corresponding to the first row.
• Figure 3: Visualization of the spectral-spatial correlation heatmap from frequency token-1 to token-5 in spectrogram of HSI.
• Figure 4: The pipeline of the frequency domain learning. (a) The structure of the spectral-wise self-attention of frequency. (b) The structure of the spectral-spatial interaction of frequency.
• Figure 5: Diagram of the overall framework. (a) Correlation-driven Mixing Domains Transformer-based Unfolding Framework. (b) The pipeline of the U-shaped prior module. (c) The correlation-driven mixing domains transformer. (d) The structure of space domain learning.