DropGaussian: Structural Regularization for Sparse-view Gaussian Splatting

TL;DR

DropGaussian addresses overfitting in sparse-view 3D Gaussian splatting by applying a prior-free regularization that randomly drops Gaussians during training to boost the visibility and gradient flow of the remaining primitives. The dropping rate is progressively increased via $r_t = \gamma \cdot \frac{t}{t_{\\mathrm{total}}}$ and the per-Gaussian opacity is compensated by $\tilde{o}_i = M(i) \cdot o_i$ with $M(i)=\frac{1}{1-r}$ to maintain per-pixel contributions. Empirical results on LLFF, Mip-NeRF360, and Blender demonstrate that DropGaussian yields competitive or superior sparse-view performance without additional priors, reducing overfitting and improving novel-view quality. The approach is simple, computationally lightweight, and easily integrable with existing 3DGS frameworks, offering a practical path to robust sparse-view rendering.

Abstract

Recently, 3D Gaussian splatting (3DGS) has gained considerable attentions in the field of novel view synthesis due to its fast performance while yielding the excellent image quality. However, 3DGS in sparse-view settings (e.g., three-view inputs) often faces with the problem of overfitting to training views, which significantly drops the visual quality of novel view images. Many existing approaches have tackled this issue by using strong priors, such as 2D generative contextual information and external depth signals. In contrast, this paper introduces a prior-free method, so-called DropGaussian, with simple changes in 3D Gaussian splatting. Specifically, we randomly remove Gaussians during the training process in a similar way of dropout, which allows non-excluded Gaussians to have larger gradients while improving their visibility. This makes the remaining Gaussians to contribute more to the optimization process for rendering with sparse input views. Such simple operation effectively alleviates the overfitting problem and enhances the quality of novel view synthesis. By simply applying DropGaussian to the original 3DGS framework, we can achieve the competitive performance with existing prior-based 3DGS methods in sparse-view settings of benchmark datasets without any additional complexity. The code and model are publicly available at: https://github.com/DCVL-3D/DropGaussian release.

Figures (7)

• Figure 1: (a) Traditional settings of 3DGS. (b) Sparse-view set tings of 3DGS. (c) Effect of DropGaussian in sparse-view settings. The rendered outputs at each viewpoint are visualized in (a), (b), and (c), respectively.
• Figure 2: The overall framework of the proposed method for 3DGS in sparse-view settings. Our DropGaussian scheme improves the visibility of Gaussians even far from the camera by randomly dropping Gaussians during the training process, thereby mitigating overfitting to the limited number of training views. In contrast, during the test phase, all the Gaussians are rendered to generate high-quality RGB images, ensuring that the complete scene representation is utilized for novel view synthesis.
• Figure 3: (a) Distribution of Gaussians having larger gradients according to the distance from camera. Note that the y-axis denotes the number of Gaussians with the gradient value, which is larger than the threshold value of densification in 3DGS. (b) Rendering results for the training view by the 3DGS. (c) Rendering results for the training view by the 3DGS with our DropGaussian scheme. (d) Rendering result for the novel view by 3DGS. (e) Rendering result for the novel view by 3DGS with our DropGaussian scheme.
• Figure 4: Changes of PSNR values during the training process on the LLFF dataset for 3DGS and 3DGS with DropGaussian. Note that rendering results for novel views at 1,000 and 10,000 iterations are also given, highlighting the intensification of overfitting in later training stages and the effectiveness of DropGaussian in mitigating this issue. The red boxes indicate regions where overfitting occurs.
• Figure 5: Results of novel view rendering on the LLFF barron22mip360 dataset. (a) Results by 3DGS kerbl233DGS. (b) Results by FSGS zhu24FSGS. (c) Results by CoR-GS zhang24CoR-GS. (d) Results by the proposed method. (e) GT image.