Dual Exposure Stereo for Extended Dynamic Range 3D Imaging

TL;DR

Robust stereo depth under extreme lighting is hampered by limited camera DR. The authors propose dual-exposure stereo, combining automatic dual exposure control (ADEC) with a motion-aware disparity estimator to extend DR for 3D imaging across alternating frames. ADEC adaptively diverges or balances exposures based on scene statistics, while dual-exposure feature fusion and RAFT-Stereo-based disparity estimation recover details in both dark and bright regions. The approach is validated on real-world robot-mounted hardware and CARLA-generated synthetic data, achieving up to 160% DR expansion with improved disparity/depth accuracy and real-time performance, outpacing existing AEC methods.

Abstract

Achieving robust stereo 3D imaging under diverse illumination conditions is an important however challenging task, due to the limited dynamic ranges (DRs) of cameras, which are significantly smaller than real world DR. As a result, the accuracy of existing stereo depth estimation methods is often compromised by under- or over-exposed images. Here, we introduce dual-exposure stereo for extended dynamic range 3D imaging. We develop automatic dual-exposure control method that adjusts the dual exposures, diverging them when the scene DR exceeds the camera DR, thereby providing information about broader DR. From the captured dual-exposure stereo images, we estimate depth using motion-aware dual-exposure stereo network. To validate our method, we develop a robot-vision system, collect stereo video datasets, and generate a synthetic dataset. Our method outperforms other exposure control methods.

Figures (10)

• Figure 1: We introduce dual-exposure stereo, a method for extended dynamic range (DR) 3D imaging. (a) We control the dual exposures synchronously set for the stereo camera to expand the effective DR of 3D imaging. From (b) the captured dual-exposure stereo images, we estimate (e) a disparity map that preserves details in both the under- and over-exposed images of the dual-exposure pair, which cannot be faithfully reconstructed in (c)&(d) one-exposure results.
• Figure 2: Image Formation. For alternating frame $i=\{1,2\}$, scene radiance $\Phi_i$, and exposure $e_i$, we simulate the captured intensity $I_i^c$ for the camera $c\in\{\text{left}, \text{right}\}$. We consider photon collection, pre- and post-gain noise, clipping, and quantization.
• Figure 3: ADEC Method Exposure Selection. The initial dual exposures are set to be the same in this example. If the scene DR is estimated as wider than the camera DR, the dual exposures diverge to capture both dark and bright regions in alternating frames.
• Figure 4: Dual Exposure Feature Fusion. For each camera $c$, we extract features $F^c_1$ and $F^c_2$, optical flow $f^c$, and weight maps $W^c_1$ and $W^c_2$ of the dual-exposure images $I^c_1$ and $I^c_2$. The second-frame features and weight map are warped to the first-frame view, and fused to the final feature $\hat{F}^c$ with the weighted summation, encoding dual-exposure information.
• Figure 5: Stereo Video Datasets. (a) Samples of our real-world dataset: sequence of stereo images and corresponding LiDAR point cloud. (b)&(c) Our imaging setup (c) mounted on a mobile robot (b), equipped with a LiDAR sensor and stereo cameras. (d) Samples of our synthetic dataset: stereo images and ground-truth disparity maps. (e) Statistics of the synthetic dataset about time, location, and scenarios.