Faster and Better 3D Splatting via Group Training

TL;DR

The paper tackles the bottleneck of training efficiency in 3D Gaussian Splatting (3DGS) due to millions of Gaussian primitives. It introduces Group Training, a plug-in framework that cyclically caches a subset of Gaussians and uses cyclic resampling, complemented by Opacity-based Prioritized Sampling (OPS) to balance densification and rendering. Empirical results across 3DGS and Mip-Splatting on multiple datasets show up to about 30% faster convergence and improved rendering quality, with OPS yielding more compact models than Random Sampling. The approach is architecture-agnostic, broadly applicable, and offers practical benefits for efficient novel view synthesis in real-world pipelines with potential for adaptive grouping in future work.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a powerful technique for novel view synthesis, demonstrating remarkable capability in high-fidelity scene reconstruction through its Gaussian primitive representations. However, the computational overhead induced by the massive number of primitives poses a significant bottleneck to training efficiency. To overcome this challenge, we propose Group Training, a simple yet effective strategy that organizes Gaussian primitives into manageable groups, optimizing training efficiency and improving rendering quality. This approach shows universal compatibility with existing 3DGS frameworks, including vanilla 3DGS and Mip-Splatting, consistently achieving accelerated training while maintaining superior synthesis quality. Extensive experiments reveal that our straightforward Group Training strategy achieves up to 30\% faster convergence and improved rendering quality across diverse scenarios. Project Website: https://chengbo-wang.github.io/3DGS-with-Group-Training/

Figures (10)

• Figure 1: Improvements and Illustrations of Applying Group Training in 3DGSkerbl3Dgaussians and Mip-SplattingYu2024MipSplatting. Grouping Training (solid line) achieves a significant increase in reconstruction speed and superior scene quality compared to baseline methods (dashed line). In the “Stump”barron2022mipnerf360 scene, Grouping Training delivers a 28% faster reconstruction while rendering leaf details with greater clarity. In the “Dr. Johnson”hedman-2018-deepblending scene, Mip-Splatting with Group Training achieves a 33% reduction in reconstruction time and effectively suppresses the generation of floating artifacts.
• Figure 2: PSNR and Time Performance of Pruning kerbl3DgaussianspapantonakisReduced3DGS and Group Training under Varying Hyperparameters. The reconstructions are performed on the "Train" scene knapitsch-2017-tanksandtemples using 3DGS kerbl3Dgaussians as the baseline. Left: The pruning method exhibits substantial instability in optimizing the trade-off between reconstruction efficiency and quality. The hyperparameter sensitivity (Opacity Threshold) presents significant challenges for optimal parameter tuning. Right: Group Training demonstrates consistent improvements in both reconstruction speed and quality, with robust performance across a wide range of hyperparameter values (Caching Ratio), enabling straightforward parameter optimization.
• Figure 3: The overall framework of Group Training. Group Training involves periodically dividing all Gaussian primitives. Specifically, at regular iteration intervals, Gaussians from all groups are merged before rendering the training view. Subsequently, all Gaussian primitives are categorized into the Under-training Group and the Caching Group according to a specified sampling strategy. Before the next grouping, the Under-training Group is utilized for Gaussian densification (Iteration 0$\sim$15K) or optimization (Iteration 15$\sim$30K), while the Caching Group remains inactive and does not participate in any calculations.
• Figure 4: Schedule for activating Group Training during reconstruction. Group Training is enabled at regular intervals. Groups Merging (Grouping without resampling) occurring at iteration $I_{\text{D}}$ and $I_{\text{O}}$ before the completion of the densification and optimization processes for global densification and optimization.
• Figure 5: The distribution of Gaussian attributes. The distribution of all Gaussian attributes (left) and those contributing specifically to densification (right) in the “Bicycle”barron2022mipnerf360 with 3DGS kerbl3Dgaussians. Top row: While the opacities are primarily concentrated around 0 and 1, the Gaussians that contribute to densification are predominantly situated around 1. Bottom row: The distribution of Volume. As densification progresses, Gaussians with lower volume become increasingly involved in the densification process.