3D Gaussian Splatting against Moving Objects for High-Fidelity Street Scene Reconstruction

TL;DR

This work proposes a novel 3D Gaussian point distribution method that introduces an adaptive transparency mechanism that eliminates moving objects while preserving high-fidelity static scene details, and integrates directional encoding with spatial position optimization to optimize storage and rendering efficiency.

Abstract

The accurate reconstruction of dynamic street scenes is critical for applications in autonomous driving, augmented reality, and virtual reality. Traditional methods relying on dense point clouds and triangular meshes struggle with moving objects, occlusions, and real-time processing constraints, limiting their effectiveness in complex urban environments. While multi-view stereo and neural radiance fields have advanced 3D reconstruction, they face challenges in computational efficiency and handling scene dynamics. This paper proposes a novel 3D Gaussian point distribution method for dynamic street scene reconstruction. Our approach introduces an adaptive transparency mechanism that eliminates moving objects while preserving high-fidelity static scene details. Additionally, iterative refinement of Gaussian point distribution enhances geometric accuracy and texture representation. We integrate directional encoding with spatial position optimization to optimize storage and rendering efficiency, reducing redundancy while maintaining scene integrity. Experimental results demonstrate that our method achieves high reconstruction quality, improved rendering performance, and adaptability in large-scale dynamic environments. These contributions establish a robust framework for real-time, high-precision 3D reconstruction, advancing the practicality of dynamic scene modeling across multiple applications.

Figures (5)

• Figure 1: The generation of 3D Gaussian distributions from an SfM sparse point cloud and their subsequent projection rendering. The black arrows represent the Operation Flow, while the blue arrows indicate the Gradient Flow. By initializing sparse SfM point clouds to generate 3D Gaussian distributions, our method effectively combines the flexibility of Gaussian distribution models with the demands of dynamic scene modeling, significantly enhancing the representation of scene geometric details and dynamic regions.
• Figure 2: Comparison of elimination of moving objects by our method and competitors in the reconstruction of the video for Nagoya scene.
• Figure 3: Comparison of elimination of moving objects by our method and competitors in reconstructing the video for Québec scene.
• Figure 4: Comparison between original images (top) and reconstructed images after removing moving objects by our method in the video for Nagoya city.
• Figure 5: Comparison between original images (top) and reconstructed images after removing moving objects in the video for Québec city. The vehicle highlighted by the green box in column (a) is a static object and thus retained, while all other vehicles and pedestrians marked by the red box in columns (b)-(e) are moving objects and eliminated by our method.