SDD-4DGS: Static-Dynamic Aware Decoupling in Gaussian Splatting for 4D Scene Reconstruction

TL;DR

SDD-4DGS introduces static-dynamic aware decoupling to Gaussian Splatting for 4D scene reconstruction by learning a per-Gaussian dynamic perception coefficient and a probabilistic dynamic/static flag. The approach integrates a progressive binary constraint and an automatic supervision signal based on uncertainty (via DINOv2) to separate dynamic and static components while maintaining geometric fidelity. Across five datasets, it achieves state-of-the-art or competitive results, with clear benefits in static detail preservation and accurate dynamic motion modeling. The work offers a practical, extensible framework for robust 4D reconstruction in real-world scenes and points to future directions in dynamic lighting and deformable objects.

Abstract

Dynamic and static components in scenes often exhibit distinct properties, yet most 4D reconstruction methods treat them indiscriminately, leading to suboptimal performance in both cases. This work introduces SDD-4DGS, the first framework for static-dynamic decoupled 4D scene reconstruction based on Gaussian Splatting. Our approach is built upon a novel probabilistic dynamic perception coefficient that is naturally integrated into the Gaussian reconstruction pipeline, enabling adaptive separation of static and dynamic components. With carefully designed implementation strategies to realize this theoretical framework, our method effectively facilitates explicit learning of motion patterns for dynamic elements while maintaining geometric stability for static structures. Extensive experiments on five benchmark datasets demonstrate that SDD-4DGS consistently outperforms state-of-the-art methods in reconstruction fidelity, with enhanced detail restoration for static structures and precise modeling of dynamic motions. The code will be released.

Figures (11)

• Figure 1: Challenges in 4D scene reconstruction: (a) Scenes typically consist of both static (e.g., buildings, walls) and dynamic (e.g., moving vehicles, pedestrians) components. (b) The red pixels represent the distribution of points projected onto the imaging plane by Gaussians. The darker the color, the more Gaussians are projected onto the pixel. As the iterative optimization proceeds, the points describing the dynamic information of the head are gradually attracted to the surrounding static area. (c) The presence of dynamic objects introduces occlusions and uneven lighting on static components, leading to artifacts and reducing the quality of static scene reconstruction. Static background depth estimation errors due to head movement. Note: Both (b) and (c) are derived from 4DGS wu20244d.
• Figure 2: Overview of the proposed SDD-4DGS pipeline for static-dynamic decoupling in 4D reconstruction. The framework decouples static and dynamic components by integrating a novel dynamic perception coefficient into 4DGS wu20244d. The pipeline involves several key stages: initialization of 3D Gaussians, computation of deformation parameters through a deformation network, and dynamic regulation using the dynamic perception coefficient. Gaussians are then decoupled into static and dynamic groups, each optimized separately through loss functions which are detailed in Sec. \ref{['subsubsec:Uncertainty Guided Decoupling']}.
• Figure 3: Visualization of optimization strategy effects on dynamic perception coefficient. From left to right: training image, results without binary constraint and self-supervision (w/o $\lambda_{bi}\mathcal{L}_{bi}$ w/o $\mathcal{L}_{asg}$), with only binary constraint (w $\lambda_{bi}\mathcal{L}_{bi}$ w/o $\mathcal{L}_{asg}$), and with both binary constraint and self-supervision (w $\lambda_{bi}\mathcal{L}_{bi}$ w $\mathcal{L}_{asg}$). The introduction of self-supervision signals significantly improves static-dynamic decoupling.
• Figure 4: Visualization of the dynamic perception coefficient distribution over training steps.
• Figure 5: Qualitative comparison between rendered results of HyperNeRF park2021hypernerf dataset. We visualize the rendering results of our method with those of other methods duan20244dwu20244dli2024spacetimeyang2023real and enlarge the local details. In addition, in order to more intuitively demonstrate our separation effect, we render the static and dynamic scenes mentioned in the method separately.