AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction

TL;DR

AutoSplat extends 3D Gaussian Splatting for autonomous driving by enforcing flat road/sky Gaussians to achieve multi-view consistency, using 3D templates and a reflected Gaussian constraint to cover unseen foreground regions, and modeling dynamic appearance with residual spherical harmonics per Gaussian. The method trains background in two phases, initializes foregrounds from realistic templates, and fuses foreground and background with per-object trajectory corrections, enabling robust novel-view synthesis including ego-vehicle lane changes. Experiments on Pandaset and KITTI demonstrate superior reconstruction quality (SSIM/LPIPS) and competitive NAS metrics, with ablations showing contributions from geometric constraints, template initialization, reflection, and dynamic appearance modeling. These components collectively enable realistic, scalable scene synthesis for safety-critical autonomous driving scenarios.

Abstract

Realistic scene reconstruction and view synthesis are essential for advancing autonomous driving systems by simulating safety-critical scenarios. 3D Gaussian Splatting excels in real-time rendering and static scene reconstructions but struggles with modeling driving scenarios due to complex backgrounds, dynamic objects, and sparse views. We propose AutoSplat, a framework employing Gaussian splatting to achieve highly realistic reconstructions of autonomous driving scenes. By imposing geometric constraints on Gaussians representing the road and sky regions, our method enables multi-view consistent simulation of challenging scenarios including lane changes. Leveraging 3D templates, we introduce a reflected Gaussian consistency constraint to supervise both the visible and unseen side of foreground objects. Moreover, to model the dynamic appearance of foreground objects, we estimate residual spherical harmonics for each foreground Gaussian. Extensive experiments on Pandaset and KITTI demonstrate that AutoSplat outperforms state-of-the-art methods in scene reconstruction and novel view synthesis across diverse driving scenarios. Visit our project page at https://autosplat.github.io/.

Figures (15)

• Figure 1: Scene reconstruction and novel view synthesis (ego-vehicle lane change) results of 3DGS 3dgs and our proposed method on a sample autonomous driving scene.
• Figure 2: AutoSplat overview. The proposed framework reconstructs the background and foreground separately and then fuses them to simulate different scenarios.
• Figure 3: Reflected Gaussian consistency in training and inference phases.
• Figure 4: Qualitative comparison of novel view synthesis (test frames) on Pandaset.
• Figure 5: Qualitative comparison of ego-vehicle lateral shift (2 meters) on Pandaset.