EfficientGS: Streamlining Gaussian Splatting for Large-Scale High-Resolution Scene Representation

TL;DR

EfficientGS tackles the Gaussian proliferation and storage burden of 3D Gaussian Splatting when representing large-scale, high-resolution scenes. It introduces three core innovations—selective Gaussian densification, Gaussian pruning, and sparse SH order increment—to reduce the Gaussian count and SH storage while maintaining rendering fidelity. The method initializes from SFM points, uses differentiable rasterization, and alternates densification, pruning, and sparse SH updates during training. Empirical results across 4K+ aerial and large-scale datasets show substantial speedups and storage reductions (about 10x smaller models) with high rendering quality, enabling near real-time rendering for expansive scenes. Overall, EfficientGS provides a scalable, efficient pipeline for high-resolution scene representation with strong generalization across diverse datasets.

Abstract

In the domain of 3D scene representation, 3D Gaussian Splatting (3DGS) has emerged as a pivotal technology. However, its application to large-scale, high-resolution scenes (exceeding 4k$\times$4k pixels) is hindered by the excessive computational requirements for managing a large number of Gaussians. Addressing this, we introduce 'EfficientGS', an advanced approach that optimizes 3DGS for high-resolution, large-scale scenes. We analyze the densification process in 3DGS and identify areas of Gaussian over-proliferation. We propose a selective strategy, limiting Gaussian increase to key primitives, thereby enhancing the representational efficiency. Additionally, we develop a pruning mechanism to remove redundant Gaussians, those that are merely auxiliary to adjacent ones. For further enhancement, we integrate a sparse order increment for Spherical Harmonics (SH), designed to alleviate storage constraints and reduce training overhead. Our empirical evaluations, conducted on a range of datasets including extensive 4K+ aerial images, demonstrate that 'EfficientGS' not only expedites training and rendering times but also achieves this with a model size approximately tenfold smaller than conventional 3DGS while maintaining high rendering fidelity.

Figures (4)

• Figure 1: Comparison with vanilla 3DGS and our EfficientGS on a large-scale high-resolution scene. The vanilla 3DGS suffers from its redundancy and thus has a low FPS and high storage pressure when representing large-scale high-resolution scenes, whereas our approach allows for a more efficient scene representation in contrast.
• Figure 2: Overview of EfficientGS. As with 3DGS, we use SFM points for initialization, and through differentiable rasterization, we can iteratively optimize the Gaussian parameters using color loss. Every few iterations, we apply a selective densification strategy(Sec. \ref{['sec3:sgd']}), where we determine whether the Gaussian reaches the steady state or not, and the non-steady state Gaussian is densified. After several densifications, we get detailed Gaussians which are fewer in number compared to vanilla 3DGS, but redundancy still exists. So we use the Gaussian pruning strategy(Sec. \ref{['sec3:gp']}) to remove auxiliary Gaussians and continue iterative optimization to hand over the work of these auxiliary Gaussians to the rest Gaussians. During the iteration process, we apply the sparse order increment strategy(Sec. \ref{['sec3:ssh']}) for the SH of Gaussians. We don't start increasing the SH order until after pruning, which greatly reduces the GPU memory pressure. Then, every few iterations, we determine the degree of need for higher-order SH for each Gaussian, depending on which we gradually increase the SH order. Finally, we get our EfficientGS model for scene representation.
• Figure 3: Gaussians in different state. The left red one is in the non-steady state and the right blue one is in the steady state. The direction of the straight line arrow represents the direction of the position gradient generated by this pixel for the Gaussian, and the length represents the norm of the gradient vector. For both Gaussians, the view-space position gradients $\sum(\vec{\nabla}_{p_I}^{g}) \approx 0$, but obviously the red one needs to be densified.
• Figure 4: Qualitative comparison on 4 scenes of Mill19 and Foothill-Town datasets. There is nearly no difference between our method and the vanilla 3DGS in terms of rendering quality, but we can achieve higher FPS and lower storage due to the more efficient scene representations our method has learned. Zoom in for more details.