Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Acquiring a Dynamic Light Field through a Single-Shot Coded Image

Ryoya Mizuno, Keita Takahashi, Michitaka Yoshida, Chihiro Tsutake, Toshiaki Fujii, Hajime Nagahara

TL;DR

This method is the first to achieve a finer temporal resolution than the camera itself in compressive light-field acquisition and designed an imaging model that synchronously applies aperture coding and pixel-wise exposure coding within a single exposure time.

Abstract

We propose a method for compressively acquiring a dynamic light field (a 5-D volume) through a single-shot coded image (a 2-D measurement). We designed an imaging model that synchronously applies aperture coding and pixel-wise exposure coding within a single exposure time. This coding scheme enables us to effectively embed the original information into a single observed image. The observed image is then fed to a convolutional neural network (CNN) for light-field reconstruction, which is jointly trained with the camera-side coding patterns. We also developed a hardware prototype to capture a real 3-D scene moving over time. We succeeded in acquiring a dynamic light field with 5x5 viewpoints over 4 temporal sub-frames (100 views in total) from a single observed image. Repeating capture and reconstruction processes over time, we can acquire a dynamic light field at 4x the frame rate of the camera. To our knowledge, our method is the first to achieve a finer temporal resolution than the camera itself in compressive light-field acquisition. Our software is available from our project webpage

Acquiring a Dynamic Light Field through a Single-Shot Coded Image

TL;DR

This method is the first to achieve a finer temporal resolution than the camera itself in compressive light-field acquisition and designed an imaging model that synchronously applies aperture coding and pixel-wise exposure coding within a single exposure time.

Abstract

We propose a method for compressively acquiring a dynamic light field (a 5-D volume) through a single-shot coded image (a 2-D measurement). We designed an imaging model that synchronously applies aperture coding and pixel-wise exposure coding within a single exposure time. This coding scheme enables us to effectively embed the original information into a single observed image. The observed image is then fed to a convolutional neural network (CNN) for light-field reconstruction, which is jointly trained with the camera-side coding patterns. We also developed a hardware prototype to capture a real 3-D scene moving over time. We succeeded in acquiring a dynamic light field with 5x5 viewpoints over 4 temporal sub-frames (100 views in total) from a single observed image. Repeating capture and reconstruction processes over time, we can acquire a dynamic light field at 4x the frame rate of the camera. To our knowledge, our method is the first to achieve a finer temporal resolution than the camera itself in compressive light-field acquisition. Our software is available from our project webpage
Paper Structure (13 sections, 6 equations, 11 figures)

This paper contains 13 sections, 6 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Computational Photography
  4. Light-Field Reconstruction
  5. Proposed Method
  6. Notations and Problem Formulation
  7. Image Acquisition Model
  8. Hardware Implementation
  9. Network Design and Training
  10. Experiments
  11. Quantitative Evaluation
  12. Experiment Using Camera Prototype
  13. Conclusions

Figures (11)

  • Figure 1: Our achievement compared with representative previous works (camera array wilburn2005high, lens-array camera ng2006digital, coded-aperture camera Inagaki_2018_ECCV, and coded exposure camera Yoshida_2020). Axes are in relative scales w.r.t. camera's spatial resolution and frame rate.
  • Figure 2: Example of dynamic light field (left) and schematic diagram of camera (right).
  • Figure 3: Coding patterns applied on aperture and pixel planes.
  • Figure 4: Example images acquired by ordinary camera Eq. (\ref{['eq:n_cam']}) (left) and our imaging model of Eq. (\ref{['eq:imaging']}) (right).
  • Figure 6: Our camera prototype (left) and optical diagram (right).
  • ...and 6 more figures