Does 3D Gaussian Splatting Need Accurate Volumetric Rendering?

TL;DR

This work analyzes whether injecting more principled volumetric rendering into 3D Gaussian Splatting (3DGS) improves performance. It derives a closed-form self-attenuation result, details a backward pass aligned with exponential attenuation, and presents an efficient ray marching pipeline with batching and adaptive binning. Through real-world (MipNeRF360) and synthetic tests, the authors compare standard 3DGS, self-attenuation variants, and OTS-based methods, finding that while accurate volumetric rendering helps at low primitive counts, the large-scale Gaussian count and optimization in 3DGS yield competitive or superior results, validating the original approximations in practical scenarios. The findings suggest that the efficiency and optimization advantages of 3DGS remain advantageous, and principled rendering improvements offer limited gains at scale on real data.

Abstract

Since its introduction, 3D Gaussian Splatting (3DGS) has become an important reference method for learning 3D representations of a captured scene, allowing real-time novel-view synthesis with high visual quality and fast training times. Neural Radiance Fields (NeRFs), which preceded 3DGS, are based on a principled ray-marching approach for volumetric rendering. In contrast, while sharing a similar image formation model with NeRF, 3DGS uses a hybrid rendering solution that builds on the strengths of volume rendering and primitive rasterization. A crucial benefit of 3DGS is its performance, achieved through a set of approximations, in many cases with respect to volumetric rendering theory. A naturally arising question is whether replacing these approximations with more principled volumetric rendering solutions can improve the quality of 3DGS. In this paper, we present an in-depth analysis of the various approximations and assumptions used by the original 3DGS solution. We demonstrate that, while more accurate volumetric rendering can help for low numbers of primitives, the power of efficient optimization and the large number of Gaussians allows 3DGS to outperform volumetric rendering despite its approximations.

Figures (51)

• Figure 1: PSNR average over all scenes
• Figure 2: PSNR average over volumetric scenes
• Figure 3: PSNR average over Nerf scenes
• Figure 4: SSIM average over all scenes
• Figure 5: SSIM average over volumetric scenes