Disparity Estimation Using a Quad-Pixel Sensor

TL;DR

A QP disparity estimation network (QPDNet), which exploits abundant QP information by fusing vertical and horizontal stereo-matching correlations for effective disparity estimation and outperforms state-of-the-art stereo and DP methods.

Abstract

A quad-pixel (QP) sensor is increasingly integrated into commercial mobile cameras. The QP sensor has a unit of 2$\times$2 four photodiodes under a single microlens, generating multi-directional phase shifting when out-focus blurs occur. Similar to a dual-pixel (DP) sensor, the phase shifting can be regarded as stereo disparity and utilized for depth estimation. Based on this, we propose a QP disparity estimation network (QPDNet), which exploits abundant QP information by fusing vertical and horizontal stereo-matching correlations for effective disparity estimation. We also present a synthetic pipeline to generate a training dataset from an existing RGB-Depth dataset. Experimental results demonstrate that our QPDNet outperforms state-of-the-art stereo and DP methods. Our code and synthetic dataset are available at https://github.com/Zhuofeng-Wu/QPDNet.

Figures (6)

• Figure 1: The overall flow of our disparity estimation. From the data captured by a QP sensor, we generate five-view stereo images, where disparities exist between a reference center image and each of the left, the right, the top, and the bottom images, according to the phase shifting principle of the DP/QP sensor in out-focus regions Okawa2019quad. Using those five-view images as inputs, our QPDNet predicts a disparity map aligned to the reference center image.
• Figure 2: The overview of our data generation process.
• Figure 3: The overall architecture of our QPDNet.
• Figure 4: Visual comparisons on the noise-free synthetic dataset.
• Figure 5: Visual comparisons on the synthetic data with Gaussian noise.