Opt3DGS: Optimizing 3D Gaussian Splatting with Adaptive Exploration and Curvature-Aware Exploitation

TL;DR

Opt3DGS tackles non-convex optimization in 3D Gaussian Splatting by decoupling training into exploration and exploitation. The exploration stage uses Adaptive Weighted SGLD to flatten the posterior and escape local optima, while the exploitation stage employs Local Quasi-Newton Direction-guided Adam to achieve curvature-aware, precise convergence. The framework refines optimization without modifying the Gaussian representation, delivering state-of-the-art rendering across benchmarks such as $\text{MipNeRF360}$, Tanks & Temples, and DeepBlending, outperforming baselines like $\text{3DGSMCMC}$ and $\text{SSS}$. Its modular, optimization-centric design suggests broad applicability to explicit differentiable rendering tasks and potential extensions to other Gaussian-based representations.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a leading framework for novel view synthesis, yet its core optimization challenges remain underexplored. We identify two key issues in 3DGS optimization: entrapment in suboptimal local optima and insufficient convergence quality. To address these, we propose Opt3DGS, a robust framework that enhances 3DGS through a two-stage optimization process of adaptive exploration and curvature-guided exploitation. In the exploration phase, an Adaptive Weighted Stochastic Gradient Langevin Dynamics (SGLD) method enhances global search to escape local optima. In the exploitation phase, a Local Quasi-Newton Direction-guided Adam optimizer leverages curvature information for precise and efficient convergence. Extensive experiments on diverse benchmark datasets demonstrate that Opt3DGS achieves state-of-the-art rendering quality by refining the 3DGS optimization process without modifying its underlying representation.

Figures (5)

• Figure 1: Exploration and Exploitation. The exploration stage promotes global search across modes using Adaptive Weighted SGLD, while the exploitation stage enables precise, curvature-aware convergence with Local Quasi-Newton direction-guided Adam optimizer.
• Figure 2: Original (a) and Flattened (b) posterior distribution. In the original distribution, High energy barriers between modes can trap the model in a single basin. The flattened distribution reduces these barriers, enabling free exploration across modes.
• Figure 3: Visualization comparison. Our method achieves higher fidelity in challenging regions like distant and fine details.
• Figure 4: PSNR results on the MipNeRF dataset with different image resolutions(red) and the maximum number of Gaussians(blue).
• Figure 5: Ablation Study about flattening coefficient $\zeta$ on the Tanks & Temples Dataset.