Mixed-granularity Implicit Representation for Continuous Hyperspectral Compressive Reconstruction

TL;DR

This work tackles the problem of long acquisition times in hyperspectral imaging by leveraging an implicit neural representation to enable continuous reconstruction across arbitrary spatial and spectral resolutions within CASSI. The proposed MGIR framework combines a Hierarchical Spectral-Spatial Implicit Encoder, a Mixed-Granularity Local Feature Aggregator, and a coordinate-based decoder to translate 2D compressed measurements into a continuous 3D hyperspectral signal, quantified as $\hat{\boldsymbol{L}} = f_{\theta}(\boldsymbol{Z}, \boldsymbol{X})$. Key innovations include the SSDW-based encoder for efficient multi-scale feature extraction and a group-wise, relative-position-aware attention mechanism for adaptive fusion across granularities. Extensive experiments on ICVL, CAVE, and KAIST demonstrate state-of-the-art spectral and spatial reconstruction across varying compression ratios, with MGIR maintaining high fidelity and offering favorable computational efficiency. The approach enables flexible display at any spatial-spectral resolution, streamlining hyperspectral imaging workflows in practical CASSI systems.

Abstract

Hyperspectral Images (HSIs) are crucial across numerous fields but are hindered by the long acquisition times associated with traditional spectrometers. The Coded Aperture Snapshot Spectral Imaging (CASSI) system mitigates this issue through a compression technique that accelerates the acquisition process. However, reconstructing HSIs from compressed data presents challenges due to fixed spatial and spectral resolution constraints. This study introduces a novel method using implicit neural representation for continuous hyperspectral image reconstruction. We propose the Mixed Granularity Implicit Representation (MGIR) framework, which includes a Hierarchical Spectral-Spatial Implicit Encoder for efficient multi-scale implicit feature extraction. This is complemented by a Mixed-Granularity Local Feature Aggregator that adaptively integrates local features across scales, combined with a decoder that merges coordinate information for precise reconstruction. By leveraging implicit neural representations, the MGIR framework enables reconstruction at any desired spatial-spectral resolution, significantly enhancing the flexibility and adaptability of the CASSI system. Extensive experimental evaluations confirm that our model produces reconstructed images at arbitrary resolutions and matches state-of-the-art methods across varying spectral-spatial compression ratios. The code will be released at https://github.com/chh11/MGIR.

Figures (13)

• Figure 1: (a) Principle of hyperspectral image Implicit Neural Representation involves encoding compressed measurements into a feature space. Given any spatial-spectral coordinate and corresponding features extracted by the encoder from the input, it decodes the spectral intensity at that coordinate. (b) Our Mixed-Granularity Implicit Representation demonstrates enhanced reconstruction quality across continuous spectral bands, outperforming the state-of-the-art fixed-band method MST cai2022mask.
• Figure 2: Schematic of the CASSI System: Continuous spectral cubes are encoded by a mask, sorted by a disperser, and subsequently compressed into a 2D measurement.
• Figure 3: (a) Overall network architecture of the proposed Mixed-Granularity Implicit Representation, featuring a Hierarchical Spectral-Spatial Implicit Encoder, a Mixed-Granularity Local Feature Aggregator, and an MLP decoder. (b) Multi-granularity Local Feature Query: The red dot represents the query point, with the dots in four distinct colors denoting four different fine-grained features. (c) Illustration of the proposed Mixed-Granularity Local Feature Aggregator.
• Figure 4: Overall architecture of the Hierarchical Spectral-Spatial Implicit Encoder (HSSIE): This encoder features a hierarchical structure composed of four Spectral-Spatial Depthwise Convolution modules (SSDW) and incorporates patch embeddings.
• Figure 5: Visualization of reconstructed results (top) from the ICVL dataset, featuring a selection of 4 out of 241 spectral bands.