Isotropic Gaussian Splatting for Real-Time Radiance Field Rendering

TL;DR

The paper addresses real-time radiance-field rendering by replacing anisotropic Gaussian kernels with scale-adaptive isotropic kernels in 3D Gaussian splatting. The approach uses a tree-based initialization and a two-stage optimization to minimize a combined $\ell_1$ and SSIM loss, achieving significant speedups without compromising geometry. Key contributions include demonstrating that isotropic kernels can be efficiently managed across multiscale representations, and that a $\sim100\times$ acceleration is achievable in practice. This work enables faster rendering and training for large-scale 3D scenes, with potential impact on 3D reconstruction, view synthesis, and dynamic object modeling.

Abstract

The 3D Gaussian splatting method has drawn a lot of attention, thanks to its high performance in training and high quality of the rendered image. However, it uses anisotropic Gaussian kernels to represent the scene. Although such anisotropic kernels have advantages in representing the geometry, they lead to difficulties in terms of computation, such as splitting or merging two kernels. In this paper, we propose to use isotropic Gaussian kernels to avoid such difficulties in the computation, leading to a higher performance method. The experiments confirm that the proposed method is about {\bf 100X} faster without losing the geometry representation accuracy. The proposed method can be applied in a large range applications where the radiance field is needed, such as 3D reconstruction, view synthesis, and dynamic object modeling.

Figures (6)

• Figure 1: The input is indicated by the red dots. The green ellipses indicate the anisotropic Gaussian kernels while the blue circles indicate the isotropic Gaussian kernels. Although anisotropic Gaussian is more representative in terms of geometry, it leads to the computation difficulties. In contrast, isotropic Gaussian is more computational efficient.
• Figure 2: (a) The input is indicated by the red dots. Green and blue lines indicate anisotropic and isotropic kernels respectively. (b) The image can be represented by size-adaptive isotropic particles. The top half is the pixel-based image and the bottom half is the particle representation. The dot size indicates the Gaussian kernel size. Although the sharp edges use more particles, the isotropic kernels lead to higher computational performance.
• Figure 3: The tree structure is adopted to initialize and organize the particles. Each cell contains one or more particles.
• Figure 4: The anisotropic and isotropic kernels in a QuadTree structure. The red is the input data. Green and blue indicate anisotropic and isotropic kernels, respectively. (a) is the initial state. (b) is a state during optimization.
• Figure 5: Anisotropic Gaussian kernels with different parameter settings. The left in each panel is the random initialization. The right is the resulting image after 2000 epochs.