G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior

TL;DR

G4Splat addresses geometry supervision gaps and multi-view inconsistencies in diffusion-prior 3D reconstructions by introducing plane-guided geometry. It derives scale-accurate depth maps from global 3D planes $\Phi_k$ and integrates plane-aware supervision into visibility estimation, novel-view planning, and inpainting with video diffusion priors. The approach demonstrates state-of-the-art geometry and appearance reconstruction across Replica, ScanNet++, and DeepBlending, with strong generalization to single-view and unposed video inputs and robust performance in unobserved regions. This plane-centric framework offers practical applicability for indoor/outdoor scenes and downstream embodied AI tasks, while maintaining memory efficiency and competitive runtime.

Abstract

Despite recent advances in leveraging generative prior from pre-trained diffusion models for 3D scene reconstruction, existing methods still face two critical limitations. First, due to the lack of reliable geometric supervision, they struggle to produce high-quality reconstructions even in observed regions, let alone in unobserved areas. Second, they lack effective mechanisms to mitigate multi-view inconsistencies in the generated images, leading to severe shape-appearance ambiguities and degraded scene geometry. In this paper, we identify accurate geometry as the fundamental prerequisite for effectively exploiting generative models to enhance 3D scene reconstruction. We first propose to leverage the prevalence of planar structures to derive accurate metric-scale depth maps, providing reliable supervision in both observed and unobserved regions. Furthermore, we incorporate this geometry guidance throughout the generative pipeline to improve visibility mask estimation, guide novel view selection, and enhance multi-view consistency when inpainting with video diffusion models, resulting in accurate and consistent scene completion. Extensive experiments on Replica, ScanNet++, and DeepBlending show that our method consistently outperforms existing baselines in both geometry and appearance reconstruction, particularly for unobserved regions. Moreover, our method naturally supports single-view inputs and unposed videos, with strong generalizability in both indoor and outdoor scenarios with practical real-world applicability. The project page is available at https://dali-jack.github.io/g4splat-web/.

Figures (10)

• Figure 1: We propose G4Splat, which integrates accurate geometry guidance with generative prior to enhance 3D scene reconstruction. Our method significantly improves geometry and appearance reconstruction quality in both observed and unobserved regions (black areas in baselines), and generalizes well across diverse scenarios, including single-view scenes and unposed videos.
• Figure 2: Overview of G4Splat. For each training loop (\ref{['sec:overall_training_strategy']}), we first extract global 3D planes from all training views and compute plane-aware depth maps (\ref{['sec:plane_modeling']}). Subsequently, we construct a visibility grid from these depth maps, select plane-aware novel views, inpaint their invisible regions, and incorporate the completed views back into the training set (\ref{['sec:geo_generative_prior']}).
• Figure 3: Visualization of intermediate results. Our method addresses key issues in prior approaches: (a) MAtCha produces noticeable errors in non-overlapping regions (highlighted by circles); (b) masks derived from alpha maps contain errors in visible areas of novel views; and (c) naive novel view selection offers only local coverage, causing visible seams in the final reconstruction.
• Figure 4: Qualitative comparison from 5 input views. Our approach achieves better appearance and geometry reconstruction with fewer Gaussian floaters in both observed and unobserved regions.
• Figure 5: Any-view scene reconstruction. Our method demonstrates strong generalization across diverse scenarios, including indoor and outdoor scenes, unposed scenes and even single-view scenes.