AbsGS: Recovering Fine Details for 3D Gaussian Splatting

TL;DR

AbsGS identifies gradient collision as the root cause of over-reconstruction in 3D Gaussian Splatting's adaptive density control and introduces a simple yet effective fix: homodirectional view-space gradients for densification. By computing a hat{g}_i as the sum of absolute per-pixel gradient magnitudes and using its L2 norm as the densification criterion, AbsGS reliably splits large Gaussians in over-reconstructed regions and recovers fine details. Extensive experiments across 13 real-world scenes show consistent improvements in SSIM and LPIPS with comparable or reduced memory, and qualitative results reveal sharper details and fewer blur artifacts. The approach is lightweight, easy to integrate with existing Gaussian Splatting methods, and will be open-sourced.

Abstract

3D Gaussian Splatting (3D-GS) technique couples 3D Gaussian primitives with differentiable rasterization to achieve high-quality novel view synthesis results while providing advanced real-time rendering performance. However, due to the flaw of its adaptive density control strategy in 3D-GS, it frequently suffers from over-reconstruction issue in intricate scenes containing high-frequency details, leading to blurry rendered images. The underlying reason for the flaw has still been under-explored. In this work, we present a comprehensive analysis of the cause of aforementioned artifacts, namely gradient collision, which prevents large Gaussians in over-reconstructed regions from splitting. To address this issue, we propose the novel homodirectional view-space positional gradient as the criterion for densification. Our strategy efficiently identifies large Gaussians in over-reconstructed regions, and recovers fine details by splitting. We evaluate our proposed method on various challenging datasets. The experimental results indicate that our approach achieves the best rendering quality with reduced or similar memory consumption. Our method is easy to implement and can be incorporated into a wide variety of most recent Gaussian Splatting-based methods. We will open source our codes upon formal publication. Our project page is available at: https://ty424.github.io/AbsGS.github.io/

Figures (8)

• Figure 1: Overview of our method. (a) The splat-based rendering technique project Gaussian $G_i$ with mean position $\mu_{i}^{3d}$ to 2D coordinate $\mu_{i}$ in pixel-space. The number of covered pixels by Gaussian $G_i$ is $m$. (b) By backpropagating, the view-space gradient $g_i$ of Gaussian $G_i$ under viewpoint $k$ is caculated as the sum of all view-space gradients of pixels that are covered by $G_i$. Since the gradients $\frac{\partial L_j}{\partial \mu_{i}}$ have different directions, the overall sum $g_i$ will have a small scale, which do not satisfy the gradient threshold for densification. (c) Motivated by above analysis, we redesign densitifaction scheme by taking the absolute value of each component $\lvert \frac{\partial L_j}{\partial \mu_{i,x}} \rvert$ and $\lvert \frac{\partial L_j}{\partial \mu_{i,y}} \rvert$ before summing. This enables to identify large Gaussians in over-reconstructed regions for split.
• Figure 2: We analyze gradient collision for view-space positional gradient, by optimizing single Gaussian to fit a image. We show the x-axis direction of pixel-wise gradient in (d), where red represents positive and green represents negative.
• Figure 3: An example to demonstrate the difference between densification strategy of 3D-GS and AbsGS. From (b) and (c), we observe that large-scale Gaussians are used to represent cement ground, which contains fine details and indeed should be represented by many small-scale Gaussians. In (d) and (e), we show the Gaussians that satisfy densification criteria of 3D-GS and ours respectively, where identified Gaussians' colors are set to white. When using $g_i$ as 3D-GS, the large Gaussians that represent cement ground are not identified while our selection strategy based on $\hat{g_i}$ can find those Gaussians.
• Figure 4: Qualitative comparisons of different methods on scenes from Mip-NeRF360barron2022mip and Tanks&Templesknapitsch2017tanks datasets. The rendering result of 3D Gaussian Splatting is blurry at regions containing high-frequency details. Our AbsGS yields significantly better rendering quality with sharper details.
• Figure 5: Comparison of identified Gaussians under different gradient thresholds. (a) The result of training the $bicycle$ scene for 3000 iterations using 3D-GS with a gradient threshold of 0.0002, showing significant over-reconstruction. (b) and (c) respectively show the selection results of 3D-GS and AbsGS with different gradient thresholds at this stage. White represents Gaussians that are selected for densification while black represents those that do not. The threshold and the number of selected Gaussians are both annotated.