D$^2$GS: Depth-and-Density Guided Gaussian Splatting for Stable and Accurate Sparse-View Reconstruction

TL;DR

The paper tackles instability in sparse-view 3D Gaussian Splatting by identifying near-field overfitting and far-field underfitting. It introduces D$^2$GS, combining Depth-and-Density Guided Dropout (DD-Drop) and Distance-Aware Fidelity Enhancement (DAFE), plus a robustness metric Inter-Model Robustness (IMR) to quantify cross-run stability. Empirical results on LLFF and Mip-NeRF360 show improved visual fidelity and significantly better robustness under sparse views, outperforming several baselines. The work advances sparse-view 3D reconstruction by coupling depth-aware regularization with depth-prior supervision and provides a practical stability measure for 3D Gaussian representations.

Abstract

Recent advances in 3D Gaussian Splatting (3DGS) enable real-time, high-fidelity novel view synthesis (NVS) with explicit 3D representations. However, performance degradation and instability remain significant under sparse-view conditions. In this work, we identify two key failure modes under sparse-view conditions: overfitting in regions with excessive Gaussian density near the camera, and underfitting in distant areas with insufficient Gaussian coverage. To address these challenges, we propose a unified framework D$^2$GS, comprising two key components: a Depth-and-Density Guided Dropout strategy that suppresses overfitting by adaptively masking redundant Gaussians based on density and depth, and a Distance-Aware Fidelity Enhancement module that improves reconstruction quality in under-fitted far-field areas through targeted supervision. Moreover, we introduce a new evaluation metric to quantify the stability of learned Gaussian distributions, providing insights into the robustness of the sparse-view 3DGS. Extensive experiments on multiple datasets demonstrate that our method significantly improves both visual quality and robustness under sparse view conditions. The project page can be found at: https://insta360-research-team.github.io/DDGS-website/.

Figures (6)

• Figure 1: Comparison of Gaussian primitives and rendered images between dense views (55 views) and sparse views (3 views) settings. Overfitting occurs in the near field (green box), while underfitting appears in the far field (red box). The number of Gaussian primitives in the corresponding field is shown below the images.
• Figure 2: The overall framework of D$^2$GS consists of a Depth-and-Density Guided Dropout (DD-Drop) module and a Distance-Aware Fidelity Enhancement (DAFE) module. The DD-Drop module adaptively removes Gaussian primitives based on depth and density indication through a dual local-global mechanism. The DAFE module enhances supervision for far-field regions using distance-aware masks.
• Figure 3: Left: The instability phenomenon of the previous method. PSNR fluctuates significantly across different training rounds, and the quality of the rendered images is highly inconsistent. Right: Calculation procedure of the IMR. The Gaussian point clouds are abstracted as Gaussian mixture distributions, and the 2-Wasserstein Distance and Optimal Transport are used.
• Figure 4: Qualitative Comparison on LLFF dataset mildenhall2019local. Comparisons were conducted with 3DGS, CoR-GS, DropGaussian. Our method effectively avoids the artifacts and maintains accurate reconstructions.
• Figure 5: Qualitative Comparison on LLFF dataset mildenhall2019local with 6-view. Comparisons were conducted with 3DGS, CoR-GS, DropGaussian. Our method effectively avoids the artifacts and maintains accurate reconstructions.