Gaussian Object Carver: Object-Compositional Gaussian Splatting with surfaces completion

TL;DR

Gaussian Object Carver (GOC) introduces an efficient, object‑compositional 3D reconstruction framework that integrates 3D Gaussian Splatting with monocular depth/normal priors and multi‑view regularization to produce high‑fidelity geometry. A zero‑shot Object Surface Completion (OSC) model then completes unobserved surfaces, yielding watertight, separable object meshes suitable for interactive editing. GOC achieves over 10× efficiency relative to state‑of‑the‑art methods while improving geometric fidelity, and OSC demonstrates strong generalization on ShapeNet/Objaverse data. This combination enables scalable digital twins and interactive applications in embodied AI, AR/VR, and simulation, where editable scene manipulation is essential.

Abstract

3D scene reconstruction is a foundational problem in computer vision. Despite recent advancements in Neural Implicit Representations (NIR), existing methods often lack editability and compositional flexibility, limiting their use in scenarios requiring high interactivity and object-level manipulation. In this paper, we introduce the Gaussian Object Carver (GOC), a novel, efficient, and scalable framework for object-compositional 3D scene reconstruction. GOC leverages 3D Gaussian Splatting (GS), enriched with monocular geometry priors and multi-view geometry regularization, to achieve high-quality and flexible reconstruction. Furthermore, we propose a zero-shot Object Surface Completion (OSC) model, which uses 3D priors from 3d object data to reconstruct unobserved surfaces, ensuring object completeness even in occluded areas. Experimental results demonstrate that GOC improves reconstruction efficiency and geometric fidelity. It holds promise for advancing the practical application of digital twins in embodied AI, AR/VR, and interactive simulation environments.

Figures (7)

• Figure 1: Invisible Surface Completion: We introduce a novel, efficient, and scalable framework for object-compositional 3D scene reconstruction, specifically designed to complete object surfaces in occluded regions. Compared to RICO li2023rico, GOC without OSC has better detail but suffers from surface holes. With the incorporation of OSC, our method generates watertight, separable object meshes, even in the presence of occlusions.
• Figure 2: Overview of GOC: Given multi-view images of a scene, we optimize 3D Gaussian Splatting (3D GS) to generate scene geometry and segmentation, applying regularization from both multi-view geometry and monocular priors. Next, incomplete objects from partially observed inputs are fed into the Object Completion Model (OSC), which performs zero-shot completion to fill in missing geometry and produce complete 3D shapes. Finally, this process yields watertight and separable object meshes, enabling flexible scene rearrangement and object-level manipulation.
• Figure 3: Illustration of OSC for surface reconstruction from sparse or incomplete point clouds. During training, points are uniformly sampled from the object’s surface mesh, filtered by specified virtual cameras, and encoded into embedding in latent space via an encoder. The decoder predicts the occupancy probability of each query point in a predefined 3D grid with the embedding. The reconstructed complete mesh is obtained by extracting the isosurface from the occupancy field.
• Figure 4: Qualitative comparison of surface reconstruction and completion quality across different methods on objects from the synthetic dataset. Zoom in for details.
• Figure 5: Qualitative comparison of reconstruction quality with (second column) and without $\mathcal{L}_{\text{IoU}}$ (third column). Using $\mathcal{L}_{\text{IoU}}$ aids OSC in establishing a well-defined isosurface threshold during the inference stage, resulting in clear and sharp mesh boundaries. Zoom in for details.