$ν$-DBA: Neural Implicit Dense Bundle Adjustment Enables Image-Only Driving Scene Reconstruction

TL;DR

$ν$-DBA addresses dense driving-scene reconstruction from RGB imagery with noisy trajectories by coupling a neural implicit 3D surface as the map with a geometric dense bundle adjustment backend driven by dense optical flow. The method uses a close-range Signed Distance Function field parameterized by a 3D hash grid and optimizes both surface and camera poses through volume rendering-based measurements, augmented by self-supervised per-scene flow refinement and optional stereo disparity cues. Through extensive experiments on KITTI-360, Waymo, and a Mine truck dataset, the approach achieves superior trajectory accuracy and denser, more detailed reconstructions compared with neural implicit baselines and traditional SLAM, while revealing how geometric cues versus photometric losses trade off in outdoor scenes. The work demonstrates a practical pathway for high-fidelity outdoor depth, surface detail, and novel-view rendering using neural implicit representations within a principled dense BA framework, with potential impact on autonomous driving perception and simulation.

Abstract

The joint optimization of the sensor trajectory and 3D map is a crucial characteristic of bundle adjustment (BA), essential for autonomous driving. This paper presents $ν$-DBA, a novel framework implementing geometric dense bundle adjustment (DBA) using 3D neural implicit surfaces for map parametrization, which optimizes both the map surface and trajectory poses using geometric error guided by dense optical flow prediction. Additionally, we fine-tune the optical flow model with per-scene self-supervision to further improve the quality of the dense mapping. Our experimental results on multiple driving scene datasets demonstrate that our method achieves superior trajectory optimization and dense reconstruction accuracy. We also investigate the influences of photometric error and different neural geometric priors on the performance of surface reconstruction and novel view synthesis. Our method stands as a significant step towards leveraging neural implicit representations in dense bundle adjustment for more accurate trajectories and detailed environmental mapping.

Figures (6)

• Figure 1: Comparison of BA methods. (a) Traditional geometric sparse BA with 3D point map. (b) Geometric dense BA with 2D keyframes. (c) Photometric dense BA with the neural implicit surface. (d) $\mathbf{\nu}$-DBA: geometric dense BA with the neural implicit surface.
• Figure 2: Overview. Our method takes a sequence of RGB images with noisy camera poses as input. In $\nu$-DBA, the self-supervised model inference optical flow and disparity as neural geometric priors, which is utilized to supervise the close-range SDF field and the camera poses with geometric losses. Moreover, a distant model and a close-range color model are used for RGB rendering.
• Figure 3: Ablation study on Self-supervised Fine-tuning. Self-supervised (ss) fine-tuning improves the details of the model and achieves a better reconstruction.
• Figure 4: Reconstruction results on the KITTI-360 dataset. Since no ground truth mesh is given, the ground truth LiDAR point cloud is displayed.
• Figure 5: Rendering results on the KITTI-360 dataset.