Hierarchical Spatial-Frequency Aggregation for Spectral Deconvolution Imaging

TL;DR

This work tackles spectral deconvolution imaging (SDI) by addressing the data-dependent, nonstationary inverse problem created by PSF engineering. It introduces a principled Hierarchical Spatial-Frequency Aggregation Unfolding Framework (HSFAUF) that decouples the SDI inverse problem into frequency-domain (linearized) and spatial-domain subproblems, leveraging diagonalization in the frequency domain to enable efficient solutions. A novel Spatial–Frequency Aggregation Transformer (SFAT) is embedded to fuse cross-domain priors, forming the Transformer-based HSFAUT. Across simulated and real experiments, HSFAUT achieves state-of-the-art reconstruction with reduced memory and computation, illustrating strong generalization across amplitude, phase, and scattering SDI systems and offering a scalable path for high-fidelity, compact spectral imaging.

Abstract

Computational spectral imaging (CSI) achieves real-time hyperspectral imaging through co-designed optics and algorithms, but typical CSI methods suffer from a bulky footprint and limited fidelity. Therefore, Spectral Deconvolution imaging (SDI) methods based on PSF engineering have been proposed to achieve high-fidelity compact CSI design recently. However, the composite convolution-integration operations of SDI render the normal-equation coefficient matrix scene-dependent, which hampers the efficient exploitation of imaging priors and poses challenges for accurate reconstruction. To tackle the inherent data-dependent operators in SDI, we introduce a Hierarchical Spatial-Spectral Aggregation Unfolding Framework (HSFAUF). By decomposing subproblems and projecting them into the frequency domain, HSFAUF transforms nonlinear processes into linear mappings, thereby enabling efficient solutions. Furthermore, to integrate spatial-spectral priors during iterative refinement, we propose a Spatial-Frequency Aggregation Transformer (SFAT), which explicitly aggregates information across spatial and frequency domains. By integrating SFAT into HSFAUF, we develop a Transformer-based deep unfolding method, \textbf{H}ierarchical \textbf{S}patial-\textbf{F}requency \textbf{A}ggregation \textbf{U}nfolding \textbf{T}ransformer (HSFAUT), to solve the inverse problem of SDI. Systematic simulated and real experiments show that HSFAUT surpasses SOTA methods with cheaper memory and computational costs, while exhibiting optimal performance on different SDI systems.

Figures (9)

• Figure 1: PSNR-Params-FLOPS comparisons with different reconstruction methods on three typical Spectral Deconvolution Imaging (SDI) systems. The vertical axis is PSNR (in dB performance), the horizontal axis is FLOPS (computational costs), and the circle radius is Params (memory costs).
• Figure 2: Depicts three paradigms of computational spectral imaging, each based on different principles. SDI offers superior integration and fidelity compared to IPM and APE.
• Figure 3: Illustration of the unified SDI reconstruction framework. By replacing the PSFs and filter functions customized for different SDI design, this framework facilitates the reconstruction of various SDI configurations.
• Figure 4: Sensing matrices and Hessian matrices of different CSI systems. The inverse process of SDI's PSFs convolution encoding can be transformed in the frequency domain into an optimization process featuring a diagonalised Hessian matrix.
• Figure 5: Illustration of HSFAUF architecture with k stages. Realize guided reconstruction by hierarchical extracting key cues from hardware a priori and imaging process characterization in the spatial and frequency domains.