PCMamba: Physics-Informed Cross-Modal State Space Model for Dual-Camera Compressive Hyperspectral Imaging

TL;DR

PCMamba tackles dual-camera compressive hyperspectral imaging by embedding the forward physical imaging process into a light, linear-complexity state-space backbone. It introduces TeX decomposition to disentangle temperature, emissivity, and texture, and a Cross-Modal Scanning Mamba Block to efficiently fuse PAN and 2D measurements with reduced sequence length. The method demonstrates state-of-the-art reconstruction quality and efficiency on simulated and real DCCHI data, with ablations confirming the contributions of TeX decomposition, CS-Mamba, and physics-consistent losses. This physics-informed approach provides both theoretical insight and practical gains for rapid, accurate HSI reconstruction in snapshot imaging systems.

Abstract

Panchromatic (PAN) -assisted Dual-Camera Compressive Hyperspectral Imaging (DCCHI) is a key technology in snapshot hyperspectral imaging. Existing research primarily focuses on exploring spectral information from 2D compressive measurements and spatial information from PAN images in an explicit manner, leading to a bottleneck in HSI reconstruction. Various physical factors, such as temperature, emissivity, and multiple reflections between objects, play a critical role in the process of a sensor acquiring hyperspectral thermal signals. Inspired by this, we attempt to investigate the interrelationships between physical properties to provide deeper theoretical insights for HSI reconstruction. In this paper, we propose a Physics-Informed Cross-Modal State Space Model Network (PCMamba) for DCCHI, which incorporates the forward physical imaging process of HSI into the linear complexity of Mamba to facilitate lightweight and high-quality HSI reconstruction. Specifically, we analyze the imaging process of hyperspectral thermal signals to enable the network to disentangle the three key physical properties-temperature, emissivity, and texture. By fully exploiting the potential information embedded in 2D measurements and PAN images, the HSIs are reconstructed through a physics-driven synthesis process. Furthermore, we design a Cross-Modal Scanning Mamba Block (CSMB) that introduces inter-modal pixel-wise interaction with positional inductive bias by cross-scanning the backbone features and PAN features. Extensive experiments conducted on both real and simulated datasets demonstrate that our method significantly outperforms SOTA methods in both quantitative and qualitative metrics.

Figures (6)

• Figure 1: Our physics-informed HSI reconstruction method.
• Figure 2: The dual-camera compressive hyperspectral imaging system.
• Figure 3: Overview of PCMamba. PCMamba consists of a state space model network with a U-net architecture composed of Cross-Modal Scanning Mamba Blocks (CSMB), and the physical synthesis process of the HSI, which combines Planck's law and TeX decomposition.
• Figure 4: Illustration of Cross-Modal Scanning Mamba Block (CSMB). CSMB performs non-overlapping cross-scanning between backbone features and PAN features to learn more compact inter-modal correlations.
• Figure 5: The visual comparisons between our method and SOTA methods on real dataset.