Efficient Perspective-Correct 3D Gaussian Splatting Using Hybrid Transparency

TL;DR

This work delivers a paradigmatic advance in real-time 3D Gaussian splatting by achieving perspective-correct rendering without matrix inversions and by deploying a per-pixel hybrid transparency scheme to stabilize depth ordering. The perspective-accurate ray-splat evaluation uses inversion-free computations based on Plücker coordinates, ensuring numerical stability across degenerate and non-degenerate Gaussians. The hybrid transparency approach splits per-pixel contributions into a depth-ordered core of K splats and a non-sorted tail, dramatically reducing global sorting overhead while mitigating popping artifacts. Evaluations on Mip-NeRF360 and Tanks & Temples show up to 2× faster training and 1.6× faster rendering with image quality on par with strong baselines, and ablations justify the chosen core size and tail handling. Collectively, the method enhances multi-view coherence and real-time performance, with strong implications for interactive radiance-field applications and potential hardware acceleration.

Abstract

3D Gaussian Splats (3DGS) have proven a versatile rendering primitive, both for inverse rendering as well as real-time exploration of scenes. In these applications, coherence across camera frames and multiple views is crucial, be it for robust convergence of a scene reconstruction or for artifact-free fly-throughs. Recent work started mitigating artifacts that break multi-view coherence, including popping artifacts due to inconsistent transparency sorting and perspective-correct outlines of (2D) splats. At the same time, real-time requirements forced such implementations to accept compromises in how transparency of large assemblies of 3D Gaussians is resolved, in turn breaking coherence in other ways. In our work, we aim at achieving maximum coherence, by rendering fully perspective-correct 3D Gaussians while using a high-quality approximation of accurate blending, hybrid transparency, on a per-pixel level, in order to retain real-time frame rates. Our fast and perspectively accurate approach for evaluation of 3D Gaussians does not require matrix inversions, thereby ensuring numerical stability and eliminating the need for special handling of degenerate splats, and the hybrid transparency formulation for blending maintains similar quality as fully resolved per-pixel transparencies at a fraction of the rendering costs. We further show that each of these two components can be independently integrated into Gaussian splatting systems. In combination, they achieve up to 2$\times$ higher frame rates, 2$\times$ faster optimization, and equal or better image quality with fewer rendering artifacts compared to traditional 3DGS on common benchmarks.

Figures (5)

• Figure 1: In 3DGS kerbl3Dgaussians, Kerbl et al. use an affine approximation for projecting 3D Gaussians. Common benchmarks do not include viewpoints where this approximation matter is highlighted. Here, we demonstrate the benefits of our perspective-correct projection by comparing renderings from 3DGS and our method.
• Figure 2: Radl et al. radl2024stopthepop discuss how 3DGS approximates depth-ordering for alpha blending, leading to popping artifacts during view rotation. To solve this without sacrificing performance, we propose a hybrid transparency approach, combining alpha blending with order-independent transparency, resulting in temporally stable rendering and an improved viewing experience.
• Figure 3: The perspectively correct screen-space bounding box of a splat (a) is given by the projection of its bounding frustum in view space (b). When transformed into local splat coordinates, the frustum planes align with tangential planes of the unit sphere (c). Our approach for splat evaluation along viewing rays makes use of the Plücker coordinate representation $( \vect{d} : \vect{m} )$. In local splat coordinates, the point along the ray that maximizes the Gaussian's value corresponds to the point $\vect{x}$ that minimizes the perpendicular distance $\|\vect{x}\|$ to the origin (d). Parts (a-c) courtesy of Weyrich et al. weyrich2007hardware; used with permission.
• Figure 4: Visual comparisons of baselines that allow for real-time rendering.
• Figure 5: Visual comparisons for different model configurations regarding our hybrid transparency approach. Using a smaller core size $K$ causes issues for reflective surfaces, as radiance fields commonly model these using semi-transparency. Disabling the order-independent tail only slightly reduces quality, especially in the sky, whereas not using it during optimization results in catastrophic failure.