Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inter and Intra Prior Learning-based Hyperspectral Image Reconstruction Using Snapshot SWIR Metasurface

Linqiang Li, Jinglei Hao, Yongqiang Zhao, Pan Liu, Haofang Yan, Ziqin Zhang, Seong G. Kong

TL;DR

This work addresses fast, high-quality SWIR hyperspectral reconstruction using a compact snapshot imaging system. It introduces a metasurface-filter coding approach with a correlation-minimizing nine-unit array and an ERRA deep unfolding network that performs inter- and intra-prior learning without sacrificing detail via adaptive feature transfer. The framework integrates a cross-stage low-rank prior learning and a three-level proximal-mapping network to enable robust spectral reconstruction across stages. On AVIRIS-NG data, the method achieves state-of-the-art PSNR and SSIM, demonstrating the potential for real-time, mobile-ready SWIR HSI with a small hardware footprint.

Abstract

Shortwave-infrared(SWIR) spectral information, ranging from 1 μm to 2.5μm, overcomes the limitations of traditional color cameras in acquiring scene information. However, conventional SWIR hyperspectral imaging systems face challenges due to their bulky setups and low acquisition speeds. This work introduces a snapshot SWIR hyperspectral imaging system based on a metasurface filter and a corresponding filter selection method to achieve the lowest correlation coefficient among these filters. This system offers the advantages of compact size and snapshot imaging. We propose a novel inter and intra prior learning unfolding framework to achieve high-quality SWIR hyperspectral image reconstruction, which bridges the gap between prior learning and cross-stage information interaction. Additionally, We design an adaptive feature transfer mechanism to adaptively transfer the contextual correlation of multi-scale encoder features to prevent detailed information loss in the decoder. Experiment results demonstrate that our method can reconstruct hyperspectral images with high speed and superior performance over existing methods.

Inter and Intra Prior Learning-based Hyperspectral Image Reconstruction Using Snapshot SWIR Metasurface

TL;DR

This work addresses fast, high-quality SWIR hyperspectral reconstruction using a compact snapshot imaging system. It introduces a metasurface-filter coding approach with a correlation-minimizing nine-unit array and an ERRA deep unfolding network that performs inter- and intra-prior learning without sacrificing detail via adaptive feature transfer. The framework integrates a cross-stage low-rank prior learning and a three-level proximal-mapping network to enable robust spectral reconstruction across stages. On AVIRIS-NG data, the method achieves state-of-the-art PSNR and SSIM, demonstrating the potential for real-time, mobile-ready SWIR HSI with a small hardware footprint.

Abstract

Shortwave-infrared(SWIR) spectral information, ranging from 1 μm to 2.5μm, overcomes the limitations of traditional color cameras in acquiring scene information. However, conventional SWIR hyperspectral imaging systems face challenges due to their bulky setups and low acquisition speeds. This work introduces a snapshot SWIR hyperspectral imaging system based on a metasurface filter and a corresponding filter selection method to achieve the lowest correlation coefficient among these filters. This system offers the advantages of compact size and snapshot imaging. We propose a novel inter and intra prior learning unfolding framework to achieve high-quality SWIR hyperspectral image reconstruction, which bridges the gap between prior learning and cross-stage information interaction. Additionally, We design an adaptive feature transfer mechanism to adaptively transfer the contextual correlation of multi-scale encoder features to prevent detailed information loss in the decoder. Experiment results demonstrate that our method can reconstruct hyperspectral images with high speed and superior performance over existing methods.
Paper Structure (14 sections, 11 equations, 9 figures, 4 tables, 1 algorithm)

This paper contains 14 sections, 11 equations, 9 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Method
  4. Problem description
  5. Filter selection
  6. Proximal mapping network
  7. Experiments
  8. Datasets
  9. Implementation Details
  10. Evaluation Indexes
  11. Comparisons with SOTA Methods
  12. Ablation Study
  13. Pattern layout analysis
  14. Conclusion

Figures (9)

  • Figure 1: Schematic representation of different snapshot compressive imaging systems. (a) The architecture of the CASSI system; (b) Our snapshot SWIR hyperspectral imaging system based on a metasurface filter.
  • Figure 2: Illustration of the proposed ERRA for HSI reconstruction, Top: the overall architecture that consists of K stages, each of which which consists of a gradient descent module and a proximal mapping module. (a) Gradient descent module; (b) Proximal mapping module.
  • Figure 3: Diagram of the $S^{2} \& l_{r}$ Prior Learning. (a) The basic unit of the $S^{2} \& l_{r}$ Prior Learning Block; (b) The structure of the spatial-spectral prior learning branch; (c) The structure of the low-rank prior learning branch; (d) The structure of the adaptive feature transfer block; (e) The component of the Fusion block; (f) The component of the FFN network.
  • Figure 4: The simulation reconstruction results of Scene03 in the test set. The results are compared with the other five methods in five different spectral bands. Residual maps are shown next to each reconstruction to facilitate comparison.
  • Figure 5: The simulation reconstruction results of Scene05 in the test set. The results are compared with the other five methods in five different spectral bands. Residual maps are shown next to each reconstruction to facilitate comparison.
  • ...and 4 more figures