Frequency-Aware Density Control via Reparameterization for High-Quality Rendering of 3D Gaussian Splatting

TL;DR

This paper proposes to establish a direct relation between density and scale through the reparameterization of the scaling parameters and ensures the consistency between them via explicit constraints, and develops a frequency-aware density control strategy, consisting of densification and deletion, to improve representation quality with fewer Gaussians.

Abstract

By adaptively controlling the density and generating more Gaussians in regions with high-frequency information, 3D Gaussian Splatting (3DGS) can better represent scene details. From the signal processing perspective, representing details usually needs more Gaussians with relatively smaller scales. However, 3DGS currently lacks an explicit constraint linking the density and scale of 3D Gaussians across the domain, leading to 3DGS using improper-scale Gaussians to express frequency information, resulting in the loss of accuracy. In this paper, we propose to establish a direct relation between density and scale through the reparameterization of the scaling parameters and ensure the consistency between them via explicit constraints (i.e., density responds well to changes in frequency). Furthermore, we develop a frequency-aware density control strategy, consisting of densification and deletion, to improve representation quality with fewer Gaussians. A dynamic threshold encourages densification in high-frequency regions, while a scale-based filter deletes Gaussians with improper scale. Experimental results on various datasets demonstrate that our method outperforms existing state-of-the-art methods quantitatively and qualitatively.

Figures (5)

• Figure 1: By linking the density and scale (the purple circle corresponds to the scale, constrained by the density), our FDS-GS can better reflect the frequency changes than 3DGS via density control (more similar to the GT's spectrum), achieving better render quality with fewer Gaussians.
• Figure 2: The pipeline of the proposed FDS-GS. FDS-GS is an improvement upon the 3DGS. By reparameterizing the scale $\mathbf{S}$ of the Gaussian and quantifying the density as $D$, we establish a direct relation between density and scale and constrain the scale according to the density to better express the frequency changes of the scene. To better control the density to achieve better rendering with fewer Gaussians, every few iterations, we compute the confidence of each Gaussian, delete the Gaussians with low confidence, and then compute a dynamic threshold for densification based on the gradient of the remaining Gaussians. Once the Gaussian distribution has changed, we will then recompute the values of $D$ and $s_a$.
• Figure 3: Confidence Calculation Example. For a given Gaussian $G$, project it onto observable images to obtain the corresponding footprints. We sample several points within the footprints to calculate the SSIM as the confidence.
• Figure 4: Qualitative comparisons of our FDS-GS method with vanilla 3DGS. Our method provides a superior restoration of scene details. GT means the ground truth. Zoom in for more details.
• Figure 5: The relation between density and scale. We randomly sample 25,000 Gaussians from the point cloud at iteration 30,000 (MipNeRF360—bicycle). The Density is computed according to Eq. (\ref{['eq:density']}), and the Scale corresponds to the volume of the Gaussian calculated using the scaling matrix. The ideal curve is obtained by fitting the result. The Gaussians of our methods are distributed around the ideal curve, while those of 3DGS show a more diffuse distribution.