3D-HGS: 3D Half-Gaussian Splatting

TL;DR

This work tackles discontinuities in 3D Gaussian Splatting by introducing 3D Half-Gaussian Splatting (3D-HGS), a plug-and-play kernel that splits each Gaussian into two halves with a splitting plane and distinct opacities. The authors derive a tractable rasterization and closed-form integrals for half-Gaussians, enabling real-time rendering with improved handling of sharp edges and texture boundaries. Across 11 scenes and multiple baselines, 3D-HGS and its variants achieve state-of-the-art novel view synthesis quality with minimal impact on rendering speed and memory, demonstrating broad applicability to existing 3D-GS pipelines. The approach advances practical 3D rendering by combining high-frequency content modeling with efficient, GPU-friendly rasterization, potentially benefiting NeRF-alternatives and real-time visualizations.

Abstract

Photo-realistic image rendering from 3D scene reconstruction has advanced significantly with neural rendering techniques. Among these, 3D Gaussian Splatting (3D-GS) outperforms Neural Radiance Fields (NeRFs) in quality and speed but struggles with shape and color discontinuities. We propose 3D Half-Gaussian (3D-HGS) kernels as a plug-and-play solution to address these limitations. Our experiments show that 3D-HGS enhances existing 3D-GS methods, achieving state-of-the-art rendering quality without compromising speed.

Figures (11)

• Figure 1: Illustration of the 3D-GS kernels and the proposed 3D Half-Gaussian kernels, where each half of the kernel is allowed to have different opacity parameters.
• Figure 2: Comparison of Half-Gaussian and Gaussian Kernels fitting a square function and their Fourier Transforms. (a): fitting a square function with 5 Gaussian kernels, and (b): fitting a square with 4 Half-Gaussian kernels. When approximating sharp edges, the Half-Gaussian kernels achieve a lower error loss (1.85) compared to Gaussian kernels (2.97). Figures (c) and (d) illustrate the Gaussian and Half-Gaussian kernels in both the spatial and frequency domains, where the Half-Gaussian demonstrates a higher bandwidth than the Gaussian kernel, indicating its superior ability to capture high-frequency components.
• Figure 3: Performance (PSNR$\uparrow$) versus rendering speed for several state-of-the-art methods kerbl20233dlu2023scaffoldkheradmand20243dbarron2021mipyu2024mip with Gaussian kernels and the proposed half-Gaussian kernels on the Mip-NeRF360 dataset barron2021mip. In all cases, using half-Gaussian kernels resulted in significant PSNR improvements, with similar or better rendering speed than the corresponding 3D Gaussian-based method.
• Figure 4: Illustration of the 3D-HG kernel, and the mapping of a pair of 3D Half-Gaussians to a 2D image.
• Figure 5: (Left) Histograms of the 3DGS opacity values and of the 3D-HGS mean opacity values of corresponding halves, trained on the Bonsai scene. The 3D-HGS mean opacity values cluster in a lower range than those in the 3DGS, implying that treating both halves identically while rendering increases the number of kernels involved in each tile, slowing down the overall process. (Right) Histogram of the differences between opacity values within a half Gaussian, normalized by the maximum opacity value within each kernel. Over 75% of the 3D-HGS kernels have normalized opacity differences over 0.5, highlighting that each half Gaussian often represents a distinct effective area in the rendering space.