Adaptive Step-size Perception Unfolding Network with Non-local Hybrid Attention for Hyperspectral Image Reconstruction

TL;DR

The paper tackles hyperspectral image reconstruction from CASSI measurements by addressing two key issues: unequal, channel-specific step sizes in iterative updates and the limitation of Transformer-based methods in balancing receptive field size with pixel-level detail. It introduces ASPUN, a FISTA-based deep unfolding network with an adaptive step-size perception module that estimates per-channel update steps, and NLIA, a non-local information aggregation framework built around the Non-local Hybrid Attention Transformer (NHAT) to maximize non-local context while preserving local details. Through extensive experiments on simulated and real data, ASPUN-NLHA achieves state-of-the-art performance, with ablations showing substantial gains from both adaptive step-size perception and NLIA components. The work advances HSI reconstruction by combining channel-aware optimization with a hybrid attention mechanism that couples global context with fine-grained texture, promising improved fidelity in practical CASSI deployments.

Abstract

Deep unfolding methods and transformer architecture have recently shown promising results in hyperspectral image (HSI) reconstruction. However, there still exist two issues: (1) in the data subproblem, most methods represents the stepsize utilizing a learnable parameter. Nevertheless, for different spectral channel, error between features and ground truth is unequal. (2) Transformer struggles to balance receptive field size with pixel-wise detail information. To overcome the aforementioned drawbacks, We proposed an adaptive step-size perception unfolding network (ASPUN), a deep unfolding network based on FISTA algorithm, which uses an adaptive step-size perception module to estimate the update step-size of each spectral channel. In addition, we design a Non-local Hybrid Attention Transformer(NHAT) module for fully leveraging the receptive field advantage of transformer. By plugging the NLHA into the Non-local Information Aggregation (NLIA) module, the unfolding network can achieve better reconstruction results. Experimental results show that our ASPUN is superior to the existing SOTA algorithms and achieves the best performance.

Figures (3)

• Figure 1: Overall architecture of our designed adaptive step-size perception unfolding network (ASPUN)
• Figure 2: Illustration of our designed Non-local Hybrid Attention Transformer.
• Figure 3: Illustration of our designed adaptive step-size perception unfolding network (ASPUN).