3D Gaussian Splatting as Markov Chain Monte Carlo

TL;DR

This work reframes 3D Gaussian Splatting as a Markov Chain Monte Carlo problem, treating Gaussians as samples from a distribution that encodes scene fidelity. By introducing stochastic gradient Langevin dynamics and a principled relocation mechanism, the method eliminates heuristic cloning/densification and enables automatic, robust optimization with adjustable Gaussian budgets. The approach yields higher rendering quality, reduced sensitivity to initialization, and better efficiency across standard NeRF-related datasets, even surpassing NeRF baselines on the MipNeRF 360 benchmark. Overall, it provides a statistically grounded, scalable framework for 3D scene representations using Gaussian splats that improves robustness and performance in neural rendering tasks.

Abstract

While 3D Gaussian Splatting has recently become popular for neural rendering, current methods rely on carefully engineered cloning and splitting strategies for placing Gaussians, which can lead to poor-quality renderings, and reliance on a good initialization. In this work, we rethink the set of 3D Gaussians as a random sample drawn from an underlying probability distribution describing the physical representation of the scene-in other words, Markov Chain Monte Carlo (MCMC) samples. Under this view, we show that the 3D Gaussian updates can be converted as Stochastic Gradient Langevin Dynamics (SGLD) updates by simply introducing noise. We then rewrite the densification and pruning strategies in 3D Gaussian Splatting as simply a deterministic state transition of MCMC samples, removing these heuristics from the framework. To do so, we revise the 'cloning' of Gaussians into a relocalization scheme that approximately preserves sample probability. To encourage efficient use of Gaussians, we introduce a regularizer that promotes the removal of unused Gaussians. On various standard evaluation scenes, we show that our method provides improved rendering quality, easy control over the number of Gaussians, and robustness to initialization.

Figures (5)

• Figure 1: Illustration of different respawn strategies -- We show a 1D example of rasterizing a Gaussian with opacity 0.95, before and after cloning them into four identical Gaussians and rasterizing them together, with different strategies. Existing methods cannot be used for MCMC as they broaden the extent of the selected Gaussian, significantly violating distribution invariance.
• Figure 2: Qualitative highlights with the same number of Gaussians -- We provide examples of novel-view rendering of 3DGS kerbl20233d and our approach on multiple scenes from different datasets (with either random or SFM initialization). We highlight the differences in inset figures. Our method faithfully represents details of the various regions thanks to our hybrid MCMC re-formulation that allows exploration without heuristics. Our results provide higher quality reconstructions. Please zoom-in to see details.
• Figure 2: Initialization ablation -- Our method provides a similar performance regardless of the initialization strategy, whereas the performance of the original 3DGS kerbl20233d differs significantly.
• Figure 3: Varying the #Gaussians -- We report the PSNR of 3DGS kerbl20233d and our method averaged over all datasets (except NeRF Synthetic).
• Figure 4: Effect of the noise term ($\boldsymbol{\epsilon}$) -- We visualize our reconstruction with (left half) and without (right half) the noise term in \ref{['eq:sgldupdt2']}. The noise terms are essential to explore the full scene extent.