DHGS: Decoupled Hybrid Gaussian Splatting for Driving Scene

TL;DR

This paper tackles high-fidelity novel view synthesis for static driving scenes by decoupling the scene into near-road and environment components represented by separate Gaussian splats. It introduces a depth-ordered hybrid renderer to fuse the two decoupled models, and leverages an implicit road surface via a pre-trained SDF to supervise road geometry, complemented by transmittance and consistency losses. The approach achieves state-of-the-art reconstruction and strong free-view synthesis performance on the Waymo dataset, demonstrating robust geometry handling under perspective changes. The work advances driving-scene neural rendering by integrating geometry priors, decoupled optimization, and depth-aware fusion to produce smooth, boundary-free imagery suitable for simulation and perception tasks.

Abstract

Existing Gaussian splatting methods often fall short in achieving satisfactory novel view synthesis in driving scenes, primarily due to the absence of crafty designs and geometric constraints for the involved elements. This paper introduces a novel neural rendering method termed Decoupled Hybrid Gaussian Splatting (DHGS), targeting at promoting the rendering quality of novel view synthesis for static driving scenes. The novelty of this work lies in the decoupled and hybrid pixel-level blender for road and non-road layers, without the conventional unified differentiable rendering logic for the entire scene. Still, consistency and continuity in superimposition are preserved through the proposed depth-ordered hybrid rendering strategy. Additionally, an implicit road representation comprised of a Signed Distance Function (SDF) is trained to supervise the road surface with subtle geometric attributes. Accompanied by the use of auxiliary transmittance loss and consistency loss, novel images with imperceptible boundary and elevated fidelity are ultimately obtained. Substantial experiments on the Waymo dataset prove that DHGS outperforms the state-of-the-art methods. The project page where more video evidences are given is: https://ironbrotherstyle.github.io/dhgs_web.

Figures (13)

• Figure 1: The comparison of the rendered road images and ellipsoids on novel view synthesis in Waymo dataset, with the top row displaying results without SDF regularizer, and the bottom row showcasing results obtained from the road model using SDF regularizer. It can be observed that the inclusion of SDF regularization leads the road model to render higher-quality images with the help of enhanced road geometry under decoupled scene representation.
• Figure 2: The training pipeline of the proposed method for driving scene reconstruction. Given consecutive multi-camera images along with their respective road and non-road masks, we initially generate decoupled road pcd (point cloud) and environment pcd, a road SDF is then pre-trained as subsequent guidance for the road Gaussian model. The environment pcd enables the initialization for the environment Gaussian model producing $I_e$, which is composed to the rendered image $I_c$ with image $I_r$ from paratactic road model via the proposed depth-ordered hybrid rendering.
• Figure 3: The diagram illustrates the proposed depth-ordered hybrid rendering strategy for the environment and road model. Corresponding primitives of each model generate pixels with colors independently through Gaussian splatting. These colors are then composited based on their rendered depths and transmittances, producing the final rendered image.
• Figure 4: Comparison of different methods on the Waymo dataset, the left column and right column display the quality of scene reconstruction and novel view synthesis respectively. Our method achieves high-quality reconstruction for both the environment and the road areas, excelling over other comparative methods.
• Figure 5: Visual comparisons on the free-view novel view synthesis. The left and right columns respectively exhibit the results of Set3 and Set4 under the free viewpoint setting, where our method significantly outperforms other comparative methods in capturing both road and environment details.