SG-Splatting: Accelerating 3D Gaussian Splatting with Spherical Gaussians

TL;DR

SG-Splatting addresses the memory and speed bottleneck of 3D Gaussian Splatting caused by 3rd-degree spherical harmonics for color. It replaces the color basis with Spherical Gaussians, using $C(d) = D + α e^{λ(d·μ - 1)}$ to model view-dependent color, and introduces multiple orthogonal SGs plus an adaptive low-degree SH mix to cover high- and low-frequency content. Per-Gaussian parameter count drops from 48 in SH to about 10 in SG, enabling smaller models and faster inference, while plug-and-play compatibility preserves integration with existing acceleration methods. Across Mip-NeRF360, Tanks&Temples, and DeepBlending, the approach yields up to 1.4–1.5x FPS improvements and roughly 46–47% storage savings with comparable image quality, signaling practical real-time applicability for scalable novel view synthesis.

Abstract

3D Gaussian Splatting is emerging as a state-of-the-art technique in novel view synthesis, recognized for its impressive balance between visual quality, speed, and rendering efficiency. However, reliance on third-degree spherical harmonics for color representation introduces significant storage demands and computational overhead, resulting in a large memory footprint and slower rendering speed. We introduce SG-Splatting with Spherical Gaussians based color representation, a novel approach to enhance rendering speed and quality in novel view synthesis. Our method first represents view-dependent color using Spherical Gaussians, instead of three degree spherical harmonics, which largely reduces the number of parameters used for color representation, and significantly accelerates the rendering process. We then develop an efficient strategy for organizing multiple Spherical Gaussians, optimizing their arrangement to achieve a balanced and accurate scene representation. To further improve rendering quality, we propose a mixed representation that combines Spherical Gaussians with low-degree spherical harmonics, capturing both high- and low-frequency color information effectively. SG-Splatting also has plug-and-play capability, allowing it to be easily integrated into existing systems. This approach improves computational efficiency and overall visual fidelity, making it a practical solution for real-time applications.

Figures (5)

• Figure 1: SG-Splatting: Spherical Gaussian-Based Splatting. The top row presents comparisons of SG-Splatting and 3D-GS on rendering results and quantitative metrics on the DeepBlending DeepBlending2018 dataset, demonstrating the ability to achieve rapid rendering. The bottom row provides zoom-in qualitative analysis of the rendered scenes, highlighting how 3D-GS exhibits noticeable artifacts, which are effectively minimized by our SG-Splatting approach.
• Figure 2: Our SG-Splatting framework for novel view synthesis. The process begins with multi-view images and corresponding camera views, which are used to initialize Gaussian primitives through SfM. These primitives are characterized by their positions, covariance matrices, opacity and color information. High-frequency color components are represented using Spherical Gaussians (SGs) to accurately capture view-dependent effects, while low-frequency color components are supplemented using diffuse or adaptive low-degree Spherical Harmonics (SH) to provide a smooth and consistent representation. The final rendered output is compared with the ground truth to evaluate the method performance.
• Figure 3: Visualization of Spherical Gaussians (SG) and how variations in $\mu$ and $\lambda$ affect the orientation and sharpness of SG. Comparing the top and bottom row highlights the differences in $\mu$, which changes the orientation of the SG. Comparing from left to right illustrates the effect of sharpness $\lambda$; as $\lambda$ increases, the SG becomes more concentrated, resulting in a sharper peak.
• Figure 4: Comparison of rendering results using different numbers of Spherical Gaussians (SGs). Using 3 SGs without orientation constraints can lead to inaccuracies (e.g., incorrectly brightening shadowed areas), while with orthogonal orientation constraints, the rendering quality is improved.Ours full further integrates adaptive low-degree SH for better rendering quality.
• Figure 5: Qualitative comparison of our SG-Splatting with baseline methods.