Pi-GS: Sparse-View Gaussian Splatting with Dense π^3 Initialization

TL;DR

This work tackles sparse-view novel view synthesis with 3D Gaussian Splatting by replacing fragile SfM-based initializations with dense geometry from a permutation-equivariant network, π^3. It fuses a PGSR-based sparse-view backbone with depth and normal supervision and introduces depth warping via pseudo-views to improve geometry alignment and reduce artifacts. A confidence-aware depth loss and a masking strategy for normals further enhance fidelity, enabling state-of-the-art performance on Tanks & Temples, LLFF, DTU, and MipNeRF360. The approach significantly mitigates floaters and view inconsistencies, offering a practical, SfM-free pathway to robust sparse-view 3D scene reconstruction, though it incurs higher memory costs and could benefit from joint pose-Gaussian optimization and diffusion priors in future work.

Abstract

Novel view synthesis has evolved rapidly, advancing from Neural Radiance Fields to 3D Gaussian Splatting (3DGS), which offers real-time rendering and rapid training without compromising visual fidelity. However, 3DGS relies heavily on accurate camera poses and high-quality point cloud initialization, which are difficult to obtain in sparse-view scenarios. While traditional Structure from Motion (SfM) pipelines often fail in these settings, existing learning-based point estimation alternatives typically require reliable reference views and remain sensitive to pose or depth errors. In this work, we propose a robust method utilizing π^3, a reference-free point cloud estimation network. We integrate dense initialization from π^3 with a regularization scheme designed to mitigate geometric inaccuracies. Specifically, we employ uncertainty-guided depth supervision, normal consistency loss, and depth warping. Experimental results demonstrate that our approach achieves state-of-the-art performance on the Tanks and Temples, LLFF, DTU, and MipNeRF360 datasets.

Figures (9)

• Figure 1: 3DGS exhibits floaters and view inconsistencies under sparse-view constraints. These artifacts are mostly caused by depth ambiguities and poor Gaussian alignment with the underlying geometry, as shown in the depth and normal maps. By incorporating depth supervision, normal supervision, and additional pseudo views, our method significantly reduces these artifacts and produces more view-consistent novel views with improved Gaussian alignment under sparse-view constraints.
• Figure 2: Comparison of the Ballroom scene from Tanks and Temples with and without opacity reset tankstemples. Background details are lost when opacity reset is executed, and image quality further degrades over the training process.
• Figure 3: Comparison between $\pi^3$ point cloud and COLMAP point cloud, of the bike scene from MipNeRF360 Dataset with 24 training images mipnerf360wang2025pi3SFM-Original.
• Figure 4: Depth rendering of the Ballroom scene from the Tanks and Temples dataset, comparing the confidence-aware Pearson loss with the standard pearson loss tankstemples. The confidence-aware loss leverages uncertainty estimates to enhance detail, particularly in the background, and also improves performance with low-resolution depth estimates.
• Figure 5: The Normal map generated from the depth map using partial derivatives, which introduces grid artifacts, and the masked normal map, which removes grid artifacts introduced by $\pi^3$ architecture wang2025pi3.