Sparse2DGS: Sparse-View Surface Reconstruction using 2D Gaussian Splatting with Dense Point Cloud
Natsuki Takama, Shintaro Ito, Koichi Ito, Hwann-Tzong Chen, Takafumi Aoki
TL;DR
Sparse2DGS tackles 3D reconstruction from only three views by leveraging dense point clouds from DUSt3R and COLMAP MVS to initialize 2D Gaussians for Gaussian Splatting. It extends 2DGS into a reconstruction pipeline that starts from a sparse SfM point cloud and ends with a TSDF-fused mesh, achieving fast optimization (~100 seconds) and high accuracy on the DTU dataset. The key idea is to fuse dense stereo-based points with MVS output to provide a robust initialization that mitigates local minima. The approach demonstrates that 3DGS-based reconstruction can match or exceed state-of-the-art methods that require large numbers of images and lengthy pretraining.
Abstract
Gaussian Splatting (GS) has gained attention as a fast and effective method for novel view synthesis. It has also been applied to 3D reconstruction using multi-view images and can achieve fast and accurate 3D reconstruction. However, GS assumes that the input contains a large number of multi-view images, and therefore, the reconstruction accuracy significantly decreases when only a limited number of input images are available. One of the main reasons is the insufficient number of 3D points in the sparse point cloud obtained through Structure from Motion (SfM), which results in a poor initialization for optimizing the Gaussian primitives. We propose a new 3D reconstruction method, called Sparse2DGS, to enhance 2DGS in reconstructing objects using only three images. Sparse2DGS employs DUSt3R, a fundamental model for stereo images, along with COLMAP MVS to generate highly accurate and dense 3D point clouds, which are then used to initialize 2D Gaussians. Through experiments on the DTU dataset, we show that Sparse2DGS can accurately reconstruct the 3D shapes of objects using just three images. The project page is available at https://gsisaoki.github.io/SPARSE2DGS/