PAGS: Priority-Adaptive Gaussian Splatting for Dynamic Driving Scenes

TL;DR

The paper tackles the efficiency-fidelity dilemma in dynamic urban scene reconstruction for autonomous driving by introducing Priority-Adaptive Gaussian Splatting (PAGS). It injects semantic priorities into both the reconstruction and rendering pipeline via Semantically-Guided Pruning and Regularization (using a Hybrid Importance Metric) and a Priority-Driven Rendering pipeline (with an Occluder Depth Pre-Pass and Early-Z culling). Key contributions include offline semantic scene decomposition, adaptive stochastic dropout, and a depth-first rendering strategy that focuses on safety-critical elements, achieving PSNR $=34.63$, SSIM $=0.933$, LPIPS $=0.073$ on Waymo, and rendering speeds over $353$ FPS with training times around $1h22m$. The practical impact is real-time, high-fidelity 3D synthesis of dynamic driving scenes with significantly reduced computation, enabling better simulation, data generation, and digital-twin applications in autonomous driving.

Abstract

Reconstructing dynamic 3D urban scenes is crucial for autonomous driving, yet current methods face a stark trade-off between fidelity and computational cost. This inefficiency stems from their semantically agnostic design, which allocates resources uniformly, treating static backgrounds and safety-critical objects with equal importance. To address this, we introduce Priority-Adaptive Gaussian Splatting (PAGS), a framework that injects task-aware semantic priorities directly into the 3D reconstruction and rendering pipeline. PAGS introduces two core contributions: (1) Semantically-Guided Pruning and Regularization strategy, which employs a hybrid importance metric to aggressively simplify non-critical scene elements while preserving fine-grained details on objects vital for navigation. (2) Priority-Driven Rendering pipeline, which employs a priority-based depth pre-pass to aggressively cull occluded primitives and accelerate the final shading computations. Extensive experiments on the Waymo and KITTI datasets demonstrate that PAGS achieves exceptional reconstruction quality, particularly on safety-critical objects, while significantly reducing training time and boosting rendering speeds to over 350 FPS.

Figures (5)

• Figure 1: Qualitative comparisons on the Waymo sun2020scalability dataset. Our method achieves a sharper and more detailed reconstruction with just over 1 hour of training. This quality surpasses both StreetGS yan2024street and EmerNeRF yang2023emernerf, which produce less detailed results despite requiring significantly longer training times of over 3h and 11h.
• Figure 2: Overview of our proposed framework. Our pipeline embeds semantic importance throughout the reconstruction and rendering process. (Left) We begin with an offline semantic scene decomposition, using foundation models to assign a semantic prior to each 3D Gaussian. (Center) During training, this semantic prior guides two synergistic optimization strategies: Adaptive Stochastic Dropout, which applies regularization based on semantic importance scores in each iteration, and Semantically-Guided Pruning, which uses a hybrid importance metric to strategically remove Gaussians at specific intervals. This yields a set of priority-optimized 3D Gaussians. (Right) At inference, a priority-driven rendering pipeline first generates a depth map using high-importance occluders, then leverages it to accelerate the final color pass through efficient hardware-based culling.
• Figure 3: Qualitative results on the Waymo dataset. The bottom row shows the final, fully converged reconstruction results for all methods. Our approach produces significantly sharper details on both dynamic objects and the static background. The top row presents a time-equalized comparison, showing the output of each method after an identical, fixed training duration. This highlights the superior convergence speed of our method, which achieves a high-fidelity result while competitors still exhibit noticeable blur and artifacts.
• Figure 4: Qualitative ablation of key components on the Waymo dataset. Our Semantically-Guided Pruning and Regularization (SPR) strategy yields a markedly sharper reconstruction of the vehicle than the baseline combining Staged Pruning (SP) and Stochastic Dropout (SD).
• Figure 5: Impact of pruning rates on PSNR and training time.