Sparse2DGS: Geometry-Prioritized Gaussian Splatting for Surface Reconstruction from Sparse Views

TL;DR

Sparse2DGS addresses the ill-posed problem of reconstructing 3D surfaces from sparse views by coupling MVS-derived geometry with Gaussian Splatting. It initializes Gaussian primitives from MVS, enforces geometry with fixed color/feature supervision, and applies a reparameterization-based disk sampling to regularize orientation and scale, complemented by a selective update strategy guided by rendered cues. The approach yields state-of-the-art or competitive results among Gaussian-based methods with significant speedups over NeRF-based methods, and robust performance on both synthetic and real-world datasets. This geometry-prioritized framework offers a practical path to accurate, complete sparse-view reconstructions in contexts where dense supervision is unavailable.

Abstract

We present a Gaussian Splatting method for surface reconstruction using sparse input views. Previous methods relying on dense views struggle with extremely sparse Structure-from-Motion points for initialization. While learning-based Multi-view Stereo (MVS) provides dense 3D points, directly combining it with Gaussian Splatting leads to suboptimal results due to the ill-posed nature of sparse-view geometric optimization. We propose Sparse2DGS, an MVS-initialized Gaussian Splatting pipeline for complete and accurate reconstruction. Our key insight is to incorporate the geometric-prioritized enhancement schemes, allowing for direct and robust geometric learning under ill-posed conditions. Sparse2DGS outperforms existing methods by notable margins while being ${2}\times$ faster than the NeRF-based fine-tuning approach.

Figures (7)

• Figure 1: Sparse2DGS boosts the strengths of Gaussian Splatting and Multi-view Stereo in sparse-view surface reconstruction, with notable improvement over 2DGS huang20242d, CLMVSNet xiong2023cl, as well as their plain combination (left). Meanwhile, Sparse2DGS is ${2} \times$ faster than NeRF-based sparse-view method long2022sparseneus (middle) and leads to complete and accurate reconstruction (right).
• Figure 2: Overview. Given sparse posed images, we first leverage the MVS points to initialize the Gaussian position $\mathbf{p}$ (Sec. \ref{['2D Gaussian Splatting with MVS initialization']}). We then leverage the MVS-derived feature for Gaussian Splatting, with fixed feature $\mathbf{f}$ and color $\mathbf{c}$ values to conduct geometrically expressive supervision and avoid appearance overfitting (Sec. \ref{['Geometrically Enhanced Supervision']}). We then optimize Gaussian primitive properties by reformulating orientation $\mathbf{R}$, scale $\mathbf{S}$, and position $\mathbf{p}$ into sampled points using the reparameterization-based disk sampling, ensuring simultaneously optimizing different forms of Gaussian properties with point-based loss (Sec. \ref{['Direct Gaussian Primitive Regularization']}). Finally, we introduce a Selective Gaussian Update strategy that leverages rendered geometry as an alternative to update Gaussian primitive positions, which further improves reconstruction quality (Sec. \ref{['Selective Gaussian Update']}).
• Figure 3: Direct Gaussian Primitive Regularization. We sample point set $\mathbf{Z}' \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$ from the standard Gaussian distribution and leverage the reparameterization technique to transform $\mathbf{Z}'$ to the point set $\mathbf{Z}$ on each Gaussian primitive. This process allows for representing Gaussian orientation $\mathbf{R}$ and scale $\mathbf{S}$ properties in the form of points easier for geometric supervision. The transformed points $\mathbf{Z}$ are then projected to the source and target views, supervised by cross-view feature consistency loss.
• Figure 4: DTU surface reconstruction results. Our method achieves more complete reconstructions with finer details.
• Figure 5: BlendedMVS reconstruction results. Our method achieves more complete and detailed reconstructions.