Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Depth from Defocus via Direct Optimization

Holly Jackson, Caleb Adams, Ignacio Lopez-Francos, Benjamin Recht

TL;DR

It is shown that alternating between convex optimization and parallel grid search can effectively solve the depth-from-defocus problem at higher resolutions than current deep learning methods.

Abstract

Though there exists a reasonable forward model for blur based on optical physics, recovering depth from a collection of defocused images remains a computationally challenging optimization problem. In this paper, we show that with contemporary optimization methods and reasonable computing resources, a global optimization approach to depth from defocus is feasible. Our approach rests on alternating minimization. When holding the depth map fixed, the forward model is linear with respect to the all-in-focus image. When holding the all-in-focus image fixed, the depth at each pixel can be computed independently, enabling embarrassingly parallel computation. We show that alternating between convex optimization and parallel grid search can effectively solve the depth-from-defocus problem at higher resolutions than current deep learning methods. We demonstrate our approach on benchmark datasets with synthetic and real defocus blur and show promising results compared to prior approaches. Our code is available at github.com/hollyjackson/dfd.

Depth from Defocus via Direct Optimization

TL;DR

It is shown that alternating between convex optimization and parallel grid search can effectively solve the depth-from-defocus problem at higher resolutions than current deep learning methods.

Abstract

Though there exists a reasonable forward model for blur based on optical physics, recovering depth from a collection of defocused images remains a computationally challenging optimization problem. In this paper, we show that with contemporary optimization methods and reasonable computing resources, a global optimization approach to depth from defocus is feasible. Our approach rests on alternating minimization. When holding the depth map fixed, the forward model is linear with respect to the all-in-focus image. When holding the all-in-focus image fixed, the depth at each pixel can be computed independently, enabling embarrassingly parallel computation. We show that alternating between convex optimization and parallel grid search can effectively solve the depth-from-defocus problem at higher resolutions than current deep learning methods. We demonstrate our approach on benchmark datasets with synthetic and real defocus blur and show promising results compared to prior approaches. Our code is available at github.com/hollyjackson/dfd.
Paper Structure (9 sections, 3 equations, 5 figures, 2 tables)

This paper contains 9 sections, 3 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Proposed method
  4. Forward model for defocus blur
  5. Direct optimization of the inverse problem
  6. Experiments
  7. Datasets
  8. Results
  9. Conclusion

Figures (5)

  • Figure 1: Example focal stack from Make3D Saxena2005Saxena2009, shown with synthetic defocus blur at five focus distances.
  • Figure 2: Illustration of a thin lens with aperture diameter $D$ and focal length $f$. Parallel rays from infinity focus at $f$ (green). A point at the focus distance ($Z_f$) forms a sharp image on the sensor at distance $s$ (blue). Points away from the focus plane form a circle of confusion with blur diameter $b$ (orange).
  • Figure 3: Qualitative results on two NYUv2 examples. Top: ground-truth AIF images and depth maps. Bottom: reconstructed AIF images and depth maps. RMSEs (left to right) are 0.0731 and 0.0230.
  • Figure 4: Qualitative results on two Make3D examples. Top: ground-truth AIF images and resized depth maps. Bottom: reconstructed AIF images and depth maps. C2 RMSEs (left to right) are 5.439 and 0.243.
  • Figure 5: Reconstructed AIF images (left) and depth maps (right) for three scenes from the mobile phone dataset Suwajanakorn2015.