Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Phase Guided Light Field for Spatial-Depth High Resolution 3D Imaging

Geyou Zhang, Ce Zhu, Kai Liu, Yipeng Liu

TL;DR

This work tackles the limitation of low spatial and depth resolution in active light-field imaging by introducing a Phase Guided Light Field (PGLF) pipeline tailored to plenoptic camera 2.0. It combines a deformed cone model (DCM) to correct axial aberrations, a phase-guided sum of absolute differences (PSAD) for robust lenslet-to-lenslet stereo matching, and a re-projection/refinement framework to reconstruct dense 3D point clouds from a single group of high-frequency fringe patterns. The approach delivers about a 10× improvement in depth-map spatial resolution (to 1280×720) while preserving high depth accuracy, enabling industrial-grade 3D measurements with few fringe patterns and feasible processing times on CPU; results on clay handicrafts and gauge-blocks demonstrate MAEs around the 0.05–0.08 mm range and phase unwrap success near 99.7%. The method paves the way for fast, dense 3D metrology with active light-field imaging, albeit with future work needed to address wrap-edge ambiguities and accelerate computation through GPU implementation.

Abstract

On 3D imaging, light field cameras typically are of single shot, and however, they heavily suffer from low spatial resolution and depth accuracy. In this paper, by employing an optical projector to project a group of single high-frequency phase-shifted sinusoid patterns, we propose a phase guided light field algorithm to significantly improve both the spatial and depth resolutions for off-the-shelf light field cameras. First, for correcting the axial aberrations caused by the main lens of our light field camera, we propose a deformed cone model to calibrate our structured light field system. Second, over wrapped phases computed from patterned images, we propose a stereo matching algorithm, i.e. phase guided sum of absolute difference, to robustly obtain the correspondence for each pair of neighbored two lenslets. Finally, by introducing a virtual camera according to the basic geometrical optics of light field imaging, we propose a reorganization strategy to reconstruct 3D point clouds with spatial-depth high resolution. Experimental results show that, compared with the state-of-the-art active light field methods, the proposed reconstructs 3D point clouds with a spatial resolution of 1280$\times$720 with factors 10$\times$ increased, while maintaining the same high depth resolution and needing merely a single group of high-frequency patterns.

Phase Guided Light Field for Spatial-Depth High Resolution 3D Imaging

TL;DR

This work tackles the limitation of low spatial and depth resolution in active light-field imaging by introducing a Phase Guided Light Field (PGLF) pipeline tailored to plenoptic camera 2.0. It combines a deformed cone model (DCM) to correct axial aberrations, a phase-guided sum of absolute differences (PSAD) for robust lenslet-to-lenslet stereo matching, and a re-projection/refinement framework to reconstruct dense 3D point clouds from a single group of high-frequency fringe patterns. The approach delivers about a 10× improvement in depth-map spatial resolution (to 1280×720) while preserving high depth accuracy, enabling industrial-grade 3D measurements with few fringe patterns and feasible processing times on CPU; results on clay handicrafts and gauge-blocks demonstrate MAEs around the 0.05–0.08 mm range and phase unwrap success near 99.7%. The method paves the way for fast, dense 3D metrology with active light-field imaging, albeit with future work needed to address wrap-edge ambiguities and accelerate computation through GPU implementation.

Abstract

On 3D imaging, light field cameras typically are of single shot, and however, they heavily suffer from low spatial resolution and depth accuracy. In this paper, by employing an optical projector to project a group of single high-frequency phase-shifted sinusoid patterns, we propose a phase guided light field algorithm to significantly improve both the spatial and depth resolutions for off-the-shelf light field cameras. First, for correcting the axial aberrations caused by the main lens of our light field camera, we propose a deformed cone model to calibrate our structured light field system. Second, over wrapped phases computed from patterned images, we propose a stereo matching algorithm, i.e. phase guided sum of absolute difference, to robustly obtain the correspondence for each pair of neighbored two lenslets. Finally, by introducing a virtual camera according to the basic geometrical optics of light field imaging, we propose a reorganization strategy to reconstruct 3D point clouds with spatial-depth high resolution. Experimental results show that, compared with the state-of-the-art active light field methods, the proposed reconstructs 3D point clouds with a spatial resolution of 1280720 with factors 10 increased, while maintaining the same high depth resolution and needing merely a single group of high-frequency patterns.
Paper Structure (15 sections, 13 equations, 9 figures, 2 tables, 1 algorithm)

This paper contains 15 sections, 13 equations, 9 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Preparation for SLF 3D Imaging
  4. Stereo Calibration
  5. Light field Calibration
  6. Calibration Procedure
  7. Initial Depth Map Estimation
  8. Phase-Guided Stereo Matching on Lenslets
  9. Initial 3D Point Clouds Computing
  10. Re-projection and Refinement Strategy for Accurate Point Clouds
  11. Experimental Results
  12. Configuration
  13. Qualitative Evaluations
  14. Quantitative Evaluations
  15. Conclusions and Future Work

Figures (9)

  • Figure 1: (a) Raw image of Lytro plenoptic camera 1.0, with $3280\times3280$ sensor resolution, $9\times9$ angular resolution, and $364\times364$ SAI resolution, (b) raw image of Raytrix plenoptic camera 2.0, with $3840\times2160$ sensor resolution, $35\times35$ angular resolution, and $110\times72$ SAI resolution and (c) effective resolution of light field imaging is associated with virtual depth $v$, which is determined by the number of lenslets that see the same target point along with the vertical or horizontal direction. The existing active light field technology can only obtain depth maps constrained by SAI resolution, far from accomplishing the effective resolution of a plenoptic camera.
  • Figure 2: (a) Measured object, (b) state-of-the-art active light fields cai2016structuredcai2018lightcai2020structured employ SAI array resulting in low spatial resolution depth map (resolution: 110$\times$71), and (c) our PGLF works on lenslet image array to have high spatial resolution depth map (resolution: 1280$\times$720).
  • Figure 3: Flowchart of our method (gray box: calibrate structure light field system with our DCM, green box: initial depth map by phase-guided light field, and blue box: re-projection and refinement strategy for dual-high resolution).
  • Figure 4: (a) Our DCM for describing axial aberration. Point clouds of the gypsum blocks: (b), (d) ground truth computed by Eq. (\ref{['EQ:Conventional3DReconst']}), (c) linear model (Eq. (\ref{['EQ:Conventional3DReconst']})), and (e) our DCM.
  • Figure 5: (a) Diagram of phase-guided stereo matching (phase map of (e), (f), and (g)). (b) and (e) depth, (c) and (f) $\phi$ and $\tilde{\phi}$, (d) and (g ) $W_0$, within two lenslet images.
  • ...and 4 more figures