Nighttime Autonomous Driving Scene Reconstruction with Physically-Based Gaussian Splatting

TL;DR

This work presents a novel approach that integrates physically based rendering into 3DGS to enhance nighttime scene reconstruction for autonomous driving and outperforms the state-of-the-art methods both quantitatively and qualitatively.

Abstract

This paper focuses on scene reconstruction under nighttime conditions in autonomous driving simulation. Recent methods based on Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) have achieved photorealistic modeling in autonomous driving scene reconstruction, but they primarily focus on normal-light conditions. Low-light driving scenes are more challenging to model due to their complex lighting and appearance conditions, which often causes performance degradation of existing methods. To address this problem, this work presents a novel approach that integrates physically based rendering into 3DGS to enhance nighttime scene reconstruction for autonomous driving. Specifically, our approach integrates physically based rendering into composite scene Gaussian representations and jointly optimizes Bidirectional Reflectance Distribution Function (BRDF) based material properties. We explicitly model diffuse components through a global illumination module and specular components by anisotropic spherical Gaussians. As a result, our approach improves reconstruction quality for outdoor nighttime driving scenes, while maintaining real-time rendering. Extensive experiments across diverse nighttime scenarios on two real-world autonomous driving datasets, including nuScenes and Waymo, demonstrate that our approach outperforms the state-of-the-art methods both quantitatively and qualitatively.

Figures (7)

• Figure 1: An overview of our approach. Our framework decomposes lighting into specular and diffuse component. Each of the respective component is modeled using the per-Gaussian ASGs and global SH module that are constrained by the BRDF
• Figure 2: Qualitative results from the Waymo Open Dataset. Columns show results from GT, Ours, OmniRe, StreetGS, and 3DGS, respectively. Fine-grained details such as traffic lights, vehicles, and trees are highlighted to illustrate differences in reconstruction quality.
• Figure 3: Qualitative results from the nuScenes Dataset. Columns show results from GT, Ours, OmniRe, StreetGS, and 3DGS, respectively. Fine-grained details such as traffic lights, vehicles, and trees are highlighted to illustrate differences in reconstruction quality.
• Figure 4: Comparison of our decomposed lighting modules. Without ASGs, fine details on vehicles and scene elements are lost due to oversmoothing. Without global SH illumination, the model fails to capture the interaction between light sources (e.g., vehicle headlights) and the surrounding environment, including shadows cast on the ground.
• Figure 5: Comparison of specular maps produced using SH and ASGs. SH-based incident specular lighting tends to create exaggerated highlights that deviate from realistic appearance, whereas ASGs capture sharper and more physically plausible specular effects.