Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multidimensional Compressed Sensing for Spectral Light Field Imaging

Wen Cao, Ehsan Miandji, Jonas Unger

TL;DR

A model that employs compressed sensing techniques to reconstruct the complete multi-spectral light field from undersampled measurements and employs a 5D basis and a novel 5D measurement model, hence, matching the intrinsic dimensionality of multispectral light fields.

Abstract

This paper considers a compressive multi-spectral light field camera model that utilizes a one-hot spectralcoded mask and a microlens array to capture spatial, angular, and spectral information using a single monochrome sensor. We propose a model that employs compressed sensing techniques to reconstruct the complete multi-spectral light field from undersampled measurements. Unlike previous work where a light field is vectorized to a 1D signal, our method employs a 5D basis and a novel 5D measurement model, hence, matching the intrinsic dimensionality of multispectral light fields. We mathematically and empirically show the equivalence of 5D and 1D sensing models, and most importantly that the 5D framework achieves orders of magnitude faster reconstruction while requiring a small fraction of the memory. Moreover, our new multidimensional sensing model opens new research directions for designing efficient visual data acquisition algorithms and hardware.

Multidimensional Compressed Sensing for Spectral Light Field Imaging

TL;DR

A model that employs compressed sensing techniques to reconstruct the complete multi-spectral light field from undersampled measurements and employs a 5D basis and a novel 5D measurement model, hence, matching the intrinsic dimensionality of multispectral light fields.

Abstract

This paper considers a compressive multi-spectral light field camera model that utilizes a one-hot spectralcoded mask and a microlens array to capture spatial, angular, and spectral information using a single monochrome sensor. We propose a model that employs compressed sensing techniques to reconstruct the complete multi-spectral light field from undersampled measurements. Unlike previous work where a light field is vectorized to a 1D signal, our method employs a 5D basis and a novel 5D measurement model, hence, matching the intrinsic dimensionality of multispectral light fields. We mathematically and empirically show the equivalence of 5D and 1D sensing models, and most importantly that the 5D framework achieves orders of magnitude faster reconstruction while requiring a small fraction of the memory. Moreover, our new multidimensional sensing model opens new research directions for designing efficient visual data acquisition algorithms and hardware.
Paper Structure (10 sections, 7 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 10 sections, 7 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Multidimensional Compressive Light Fields
  4. The measurement model
  5. Compressed sensing
  6. Learning-based multidimensional dictionary
  7. Multidimensional measurement model
  8. Reconstruction
  9. Evaluation and results
  10. Conclusions

Figures (4)

  • Figure 1: Illustrates the proposed compressed sensing framework for multi-spectral light field capture and reconstruction. Compressed light fields are captured using a lenslet array placed in the optical path and a one-hot spectral CFA mask placed on the sensor. Our proposed $nD$ compressed sensing formulation improves the reconstruction time by orders of magnitude as compared to the commonly used 1D compressed sensing techniques without any quality degradation.
  • Figure 2: Illustration of Kronecker-based coded attenuation masks.
  • Figure 3: 5D CS Performance comparison (5D DCT, 5D AMDE) of reconstruction results using different number of snapshots of Elephant scene. Captions are identical to Fig. \ref{['figss']}.
  • Figure 4: Performance comparison of Reconstruction Results of 5D DCT, 1D AMDE and 5D AMDE of five test spectral light fields using three snapshots. Evaluation metric PSNR in dB. The multi-spectral channels are converted to RGB according to CIE 1913 and CIE D65. The images are chosen as the angular image [2 3] out of the $5\times5$ reconstructed views.