Taming 3DGS: High-Quality Radiance Fields with Limited Resources

TL;DR

This work tackles the resource-intensiveness of 3D Gaussian Splatting (3DGS) by introducing a budgeted, purely constructive densification pipeline. A parabolic growth schedule guarantees an exact final number of Gaussians, while a score-based, steerable densification guides additions to high-impact regions; high-opacity Gaussians further boost expressiveness without ballooning the primitive count. The authors also deliver substantial runtime optimizations, including per-splat backpropagation, batched SH updates, and faster differentiable loss computation, achieving 4–5× training-time reductions and enabling training on consumer hardware with fixed-size models. In budgeted regimes, the method attains competitive rendering quality and, with larger budgets, surpasses 3DGS, enabling practical novel-view synthesis on constrained devices such as mobile or edge platforms. The approach complements existing 3DGS compression techniques and opens avenues for latency-constrained streaming and region-prioritized rendering in real-world applications.

Abstract

3D Gaussian Splatting (3DGS) has transformed novel-view synthesis with its fast, interpretable, and high-fidelity rendering. However, its resource requirements limit its usability. Especially on constrained devices, training performance degrades quickly and often cannot complete due to excessive memory consumption of the model. The method converges with an indefinite number of Gaussians -- many of them redundant -- making rendering unnecessarily slow and preventing its usage in downstream tasks that expect fixed-size inputs. To address these issues, we tackle the challenges of training and rendering 3DGS models on a budget. We use a guided, purely constructive densification process that steers densification toward Gaussians that raise the reconstruction quality. Model size continuously increases in a controlled manner towards an exact budget, using score-based densification of Gaussians with training-time priors that measure their contribution. We further address training speed obstacles: following a careful analysis of 3DGS' original pipeline, we derive faster, numerically equivalent solutions for gradient computation and attribute updates, including an alternative parallelization for efficient backpropagation. We also propose quality-preserving approximations where suitable to reduce training time even further. Taken together, these enhancements yield a robust, scalable solution with reduced training times, lower compute and memory requirements, and high quality. Our evaluation shows that in a budgeted setting, we obtain competitive quality metrics with 3DGS while achieving a 4--5x reduction in both model size and training time. With more generous budgets, our measured quality surpasses theirs. These advances open the door for novel-view synthesis in constrained environments, e.g., mobile devices.

Figures (6)

• Figure 1: Our method makes 3DGS optimization fast and flexible, achieving high rendering quality on a budget. Left and middle: model size and training time are reduced by more than $5\times$. Right: Our method produces models with an exact, user-specified target size, surpassing 3DGS quality as the target increases.
• Figure 2: Overview of our method. (a) We propose a systematic redesign of 3DGS densification. To select Gaussians to densify, we sample training views and compute per-pixel saliency. A scoring function $F$ combines gradient, saliency, and primitive properties into a per-Gaussian score $\textbf{S}_g$. (b) The addition of new Gaussians follows a predictable schedule. We follow a growth curve that mimics 3DGS' behavior and can be fitted to yield any desired model size after training.
• Figure 3: (a) Time spent in different parts (forward pass, backward pass, optimizer step) of one 3DGS iteration in four scenes (garden, bicycle, kitchen, bonsai). Left: analysis of original 3DGS at different stages of training. Right: using our budgeted densification and performance optimizations. (b) Gradient backpropagation. (Top) 3DGS utilizes per-pixel parallelization for backpropagation. Atomic gradient additions create frequent collisions, slowing down the backward. Instead, we parallelize on the projected 2D splats, such that each thread (and pixel) contributes to one Gaussian at a time. (Bottom) The gradient calculation requires processing a set of per-pixel, per-splat values resulting in an implicit traversal of a splat $\iff$ pixel state table. During the forward, we store the pixel states for every $32^{\text{nd}}$ splat in the sorted list. For the backward, we divide the splats into buckets of size $32$, each of which gets scheduled to a CUDA warp. Warps use intra-warp shuffling to produce their share of the state table cheaply.
• Figure 4: Backward pass duration in training of Bicycle using 3DGS, DISTWAR distwar and our variants. For our approach, we plot the times when used with original 3DGS densification and with our more tightly budgeted schedule.
• Figure 5: Qualitative comparison of results produced with our method in two budgeted scenarios to 3DGS, as well as Instant-NGP, whose training times match those of Ours. While the strictly budgeted scenario produces highly competitive results, a higher budget resolves occasional remaining blurry Gaussians.