A Step to Decouple Optimization in 3DGS

TL;DR

This work identifies two optimization couplings in 3D Gaussian Splatting (3DGS) that hinder efficiency and regularization control. It proposes a decoupled framework—Sparse Adam, Re-State Regularization (RSR), and Decoupled Attribute Regularization (DAR)—and then recouples them into AdamW-GS to improve both speed and reconstruction quality. Empirical results across 3DGS and 3DGS-MCMC show reduced redundancy, better visual fidelity, and substantial runtime savings, with robust ablations validating the roles of each component. The approach preserves or enhances performance across multiple datasets and pipelines, while enabling controllable regularization and exploration. This work offers a principled path to more efficient optimization for explicit 3D scene representations and extends practical utility for real-time view synthesis scenarios.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a powerful technique for real-time novel view synthesis. As an explicit representation optimized through gradient propagation among primitives, optimization widely accepted in deep neural networks (DNNs) is actually adopted in 3DGS, such as synchronous weight updating and Adam with the adaptive gradient. However, considering the physical significance and specific design in 3DGS, there are two overlooked details in the optimization of 3DGS: (i) update step coupling, which induces optimizer state rescaling and costly attribute updates outside the viewpoints, and (ii) gradient coupling in the moment, which may lead to under- or over-effective regularization. Nevertheless, such a complex coupling is under-explored. After revisiting the optimization of 3DGS, we take a step to decouple it and recompose the process into: Sparse Adam, Re-State Regularization and Decoupled Attribute Regularization. Taking a large number of experiments under the 3DGS and 3DGS-MCMC frameworks, our work provides a deeper understanding of these components. Finally, based on the empirical analysis, we re-design the optimization and propose AdamW-GS by re-coupling the beneficial components, under which better optimization efficiency and representation effectiveness are achieved simultaneously.

Figures (10)

• Figure 1: a: The $\sqrt{v}(o)$ in 3DGS-MCMC with the different optimizer. More examples can be found in Appendix Figure \ref{['plot::opacity_moment_adainfo_denorm']}. b: the opacity regularization decisive term in 3DGS-MCMC with the different optimizer. c-d: The average and max magnitude of $m(o)/\sqrt{v(o)}$ in every iteration.
• Figure 2: a-d: The primitive number change during training in 4 methods (Vanilla 3DGS, Redundant Primitivs Removal/RePR, MaskGaussion and 3DGS with Our proposed AdamW-GS). e: illustrates the iteration ranges over which different components affect the primitives number.
• Figure 3: a-d: Reconstruction results visualization. More can be found in Appendix Sec.\ref{['sec::all_results']}. e-f: The Reallocated Primitive Number in 3DGS-MCMC Framework. For outdoor scenes, MC17 and MC8 differ only in the StSS sampling ratio, where MC8(StSSMC3)$>$MC17(StSSMC1)=MCMC-Sparse-RSR. For indoor scenes, MC8 uses StSSMC1. More information can be checked in Table \ref{['tab::mipnerf360_basic']}.
• Figure 4: Training Time cost Comparison (Vanilla Ours and MCMC Ours denote 3DGS+AdamW-GS and 3DGSMCMC+AdamW-GS).
• Figure 5: The average of the second moment in valid primitives.