Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dense Point Clouds Matter: Dust-GS for Scene Reconstruction from Sparse Viewpoints

Shan Chen, Jiale Zhou, Lei Li

TL;DR

Dust-GS introduces an innovative point cloud initialization technique that remains effective even with sparse input data, and leverages a hybrid strategy that integrates an adaptive depth-based masking technique, thereby enhancing the accuracy and detail of reconstructed scenes.

Abstract

3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in scene synthesis and novel view synthesis tasks. Typically, the initialization of 3D Gaussian primitives relies on point clouds derived from Structure-from-Motion (SfM) methods. However, in scenarios requiring scene reconstruction from sparse viewpoints, the effectiveness of 3DGS is significantly constrained by the quality of these initial point clouds and the limited number of input images. In this study, we present Dust-GS, a novel framework specifically designed to overcome the limitations of 3DGS in sparse viewpoint conditions. Instead of relying solely on SfM, Dust-GS introduces an innovative point cloud initialization technique that remains effective even with sparse input data. Our approach leverages a hybrid strategy that integrates an adaptive depth-based masking technique, thereby enhancing the accuracy and detail of reconstructed scenes. Extensive experiments conducted on several benchmark datasets demonstrate that Dust-GS surpasses traditional 3DGS methods in scenarios with sparse viewpoints, achieving superior scene reconstruction quality with a reduced number of input images.

Dense Point Clouds Matter: Dust-GS for Scene Reconstruction from Sparse Viewpoints

TL;DR

Dust-GS introduces an innovative point cloud initialization technique that remains effective even with sparse input data, and leverages a hybrid strategy that integrates an adaptive depth-based masking technique, thereby enhancing the accuracy and detail of reconstructed scenes.

Abstract

3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in scene synthesis and novel view synthesis tasks. Typically, the initialization of 3D Gaussian primitives relies on point clouds derived from Structure-from-Motion (SfM) methods. However, in scenarios requiring scene reconstruction from sparse viewpoints, the effectiveness of 3DGS is significantly constrained by the quality of these initial point clouds and the limited number of input images. In this study, we present Dust-GS, a novel framework specifically designed to overcome the limitations of 3DGS in sparse viewpoint conditions. Instead of relying solely on SfM, Dust-GS introduces an innovative point cloud initialization technique that remains effective even with sparse input data. Our approach leverages a hybrid strategy that integrates an adaptive depth-based masking technique, thereby enhancing the accuracy and detail of reconstructed scenes. Extensive experiments conducted on several benchmark datasets demonstrate that Dust-GS surpasses traditional 3DGS methods in scenarios with sparse viewpoints, achieving superior scene reconstruction quality with a reduced number of input images.
Paper Structure (13 sections, 9 equations, 3 figures, 2 tables)

This paper contains 13 sections, 9 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. method
  3. 3D Gaussian Splatting
  4. Initialize point cloud
  5. Depth Correlation Loss
  6. Dynamic depth mask
  7. Loss Function
  8. EXPERIMENTAL RESULTS
  9. Datasets and Evaluation Metrics
  10. Implementation Details
  11. Qualitative and Quantitative Evaluation
  12. Ablations and Analysis
  13. Conclusion

Figures (3)

  • Figure 1: Novel View Synthesis Comparison. Comparison of 3DGS kerbl20233d and Mip-Splatting yu2024mip with our Dust-GS shows that Dust-GS outperforms the other methods in synthesizing close-up scenes.
  • Figure 2: The framework estimates camera poses and registers point clouds using DUSt3R, initializes 3D Gaussian primitives, and optimizes them with RGB, depth, Gaussian Process Priors (GPP), and dynamic depth masks for improved scene reconstruction.
  • Figure 3: Qualitative results on the MipNeRF360 and BungeeNeRF datasets. Ours(Dust-GS) achieves superior geometric consistency and detail fidelity compared to 3DGS kerbl20233d, SparseGS xiong2023sparsegs, and Mip-Splatting yu2024mip, with results that are closer to the ground truth (GT).