Light-Field Dataset for Disparity Based Depth Estimation

TL;DR

This work tackles the shortage of suitable light-field datasets for disparity-based depth estimation by introducing a comprehensive real-and-synthetic LF dataset using a Lytro Illum camera and Blender, with ground-truth depth for selected scenes. It examines how focal distance affects disparity and the angular-spatial trade-off in micro-lens based LF cameras, and shows that existing public datasets fail to preserve the inverse depth-disparity relationship. The dataset comprises 285 real LF images, 13 synthetic LF images, plus a synthetic stereo LF dataset, and includes calibration data and ground-truth depths to enable robust benchmarking. The resource enables testing of disparity-based depth methods and related LF tasks across realistic and synthetic scenarios, with an accompanying quantitative benchmark.

Abstract

A Light Field (LF) camera consists of an additional two-dimensional array of micro-lenses placed between the main lens and sensor, compared to a conventional camera. The sensor pixels under each micro-lens receive light from a sub-aperture of the main lens. This enables the image sensor to capture both spatial information and the angular resolution of a scene point. This additional angular information is used to estimate the depth of a 3-D scene. The continuum of virtual viewpoints in light field data enables efficient depth estimation using Epipolar Line Images (EPIs) with robust occlusion handling. However, the trade-off between angular information and spatial information is very critical and depends on the focal position of the camera. To design, develop, implement, and test novel disparity-based light field depth estimation algorithms, the availability of suitable light field image datasets is essential. In this paper, a publicly available light field image dataset is introduced and thoroughly described. We have also demonstrated the effect of focal position on the disparity of a 3-D point as well as the shortcomings of the currently available light field dataset. The proposed dataset contains 285 light field images captured using a Lytro Illum LF camera and 13 synthetic LF images. The proposed dataset also comprises a synthetic dataset with similar disparity characteristics to those of a real light field camera. A real and synthetic stereo light field dataset is also created by using a mechanical gantry system and Blender. The dataset is available at https://github.com/aupendu/light-field-dataset.

Figures (18)

• Figure 1: Graphical description of our proposed dataset
• Figure 2: Heidelberg New and Old Synthetic Benchmark Dataset. Top to bottom -Still life, Tower and Town. (a) left view, (b) right view, (c) point correspondences, (d) refined point correspondences, (e) Color markers where the blue color represents key points with a negative disparity and the yellow color represents the positive disparity.
• Figure 3:
• Figure 4: Our Real Data-set. Top to bottom - Scene 1, Scene 2, Scene 3 and Scene 4. (a) left view, (b) right view, (c) point correspondences, (d) refined point correspondences, (e) correspondences with positive disparity, (f) Color markers, blue color for key points having negative disparity, yellow color for the positive disparity.
• Figure 5: Heidelberg New and Old Benchmark Datasets: disparity distribution over the key-points when disparity is greater than 2. (a) Buddha, (b) Buddha2, (c) Still Life, (d) Tower, (e) Town.