SS3DM: Benchmarking Street-View Surface Reconstruction with a Synthetic 3D Mesh Dataset

TL;DR

The SS3DM dataset is introduced, comprising precise mesh models exported from the CARLA simulator that facilitate accurate position evaluation and include normal vectors for evaluating surface normal in 3D reconstruction.

Abstract

Reconstructing accurate 3D surfaces for street-view scenarios is crucial for applications such as digital entertainment and autonomous driving simulation. However, existing street-view datasets, including KITTI, Waymo, and nuScenes, only offer noisy LiDAR points as ground-truth data for geometric evaluation of reconstructed surfaces. These geometric ground-truths often lack the necessary precision to evaluate surface positions and do not provide data for assessing surface normals. To overcome these challenges, we introduce the SS3DM dataset, comprising precise \textbf{S}ynthetic \textbf{S}treet-view \textbf{3D} \textbf{M}esh models exported from the CARLA simulator. These mesh models facilitate accurate position evaluation and include normal vectors for evaluating surface normal. To simulate the input data in realistic driving scenarios for 3D reconstruction, we virtually drive a vehicle equipped with six RGB cameras and five LiDAR sensors in diverse outdoor scenes. Leveraging this dataset, we establish a benchmark for state-of-the-art surface reconstruction methods, providing a comprehensive evaluation of the associated challenges. For more information, visit our homepage at https://ss3dm.top.

Figures (13)

• Figure 1: Overview of SS3DM: A 3D mesh dataset for benchmarking surface reconstruction of street-view outdoor scenes. a) High-fidelity 3D mesh models are provided for accurate geometric evaluation. b) SS3DM contains multi-view RGB video sequences which can be used as inputs for 3D surface reconstruction, along with depth and semantic information. c) Multi-view LiDAR points are also included as auxiliary inputs for 3D reconstruction. d) The street-view sequences are collected from the CARLA simulator with on-car sensors.
• Figure 2: Geometric ground-truths in Waymo (LiDAR points) and the proposed SS3DM (meshes).
• Figure 3: Specifications for the on-car sensors.
• Figure 4: Camera and LiDAR locations.
• Figure 5: We collect our sequences in eight towns, including different types of areas and buildings. For each scene, we present a front camera image (left) and a bird's-eye view of the entire town (right).