GS-DMSR: Dynamic Sensitive Multi-scale Manifold Enhancement for Accelerated High-Quality 3D Gaussian Splatting

TL;DR

This work tackles the challenge of balancing rapid convergence with high‑fidelity rendering in dynamic 3D reconstruction by introducing GS‑DMSR. The framework combines Motion Saliency‑Driven Dynamic Gaussian Optimization (MS‑DGO), which adaptively focuses gradient updates on motion‑salient Gaussians, with a Multi‑scale Disentangled Manifold Deformation module that couples an implicit nonlinear decoder to an explicit deformation field. This synergy accelerates training, reduces storage overhead, and strengthens modeling of complex deformations, while maintaining real‑time rendering capability (e.g., up to 96 FPS on synthetic data). Across synthetic and real‑world datasets, GS‑DMSR achieves state‑of‑the‑art quality with improved convergence efficiency, offering a practical pathway for high‑quality, dynamic 3D Gaussian Splatting.

Abstract

In the field of 3D dynamic scene reconstruction, how to balance model convergence rate and rendering quality has long been a critical challenge that urgently needs to be addressed, particularly in high-precision modeling of scenes with complex dynamic motions. To tackle this issue, this study proposes the GS-DMSR method. By quantitatively analyzing the dynamic evolution process of Gaussian attributes, this mechanism achieves adaptive gradient focusing, enabling it to dynamically identify significant differences in the motion states of Gaussian models. It then applies differentiated optimization strategies to Gaussian models with varying degrees of significance, thereby significantly improving the model convergence rate. Additionally, this research integrates a multi-scale manifold enhancement module, which leverages the collaborative optimization of an implicit nonlinear decoder and an explicit deformation field to enhance the modeling efficiency for complex deformation scenes. Experimental results demonstrate that this method achieves a frame rate of up to 96 FPS on synthetic datasets, while effectively reducing both storage overhead and training time.Our code and data are available at https://anonymous.4open.science/r/GS-DMSR-2212.

Figures (7)

• Figure 1: Motion Saliency-Driven Dynamic Gaussian Optimization.For each Gaussian, we quantify the dynamic variation properties of its attributes and classify them into Gaussians with different saliency levels. We then label these Gaussians and optimize the high-saliency ones.
• Figure 2: The overall pipeline of our model. For a set of 3D Gaussians $G$, we extract the center coordinates and timestamp $t$ of each Gaussian by querying multi-resolution voxel planes; using $MS-DGO$, we distinguish Gaussians with different saliency coefficients and optimize those with high saliency coefficients; next, we compute the voxel features; decoding these features via a miniaturized multi-head Gaussian deformation decoder yields the deformed 3D Gaussian $G^{\prime}$ at timestamp t; finally, applying Gaussian splatting to the deformed Gaussians generates the final rendered image.
• Figure 3: For the synthetic dataset, visualization comparison experiments with other models were conducted wu20244dgan2023v4dfang2022fast. The rendering results retain the default green background, and this study adopts the rendering parameter configurations of wu20244d.
• Figure 4: Visualization of the HyperNeRF park2021hypernerf dataset compared with other methodsgan2023v4dfang2022fastwu20244d . ‘GT’ stands for ground truth images.Please zoom in for better observation.
• Figure 5: Experimental results show that in the visualization results, our method can recover more details compared to other methods kerbl20233dpark2021hypernerfwu20244d.