LaS-Comp: Zero-shot 3D Completion with Latent-Spatial Consistency

TL;DR

This paper introduces LaS-Comp, a zero-shot and category-agnostic approach that leverages the rich geometric priors of 3D foundation models to enable 3D shape completion across diverse types of partial observations and introduces Omni-Comp, a comprehensive benchmark combining real-world and synthetic data with diverse and challenging partial patterns.

Abstract

This paper introduces LaS-Comp, a zero-shot and category-agnostic approach that leverages the rich geometric priors of 3D foundation models to enable 3D shape completion across diverse types of partial observations. Our contributions are threefold: First, \ourname{} harnesses these powerful generative priors for completion through a complementary two-stage design: (i) an explicit replacement stage that preserves the partial observation geometry to ensure faithful completion; and (ii) an implicit refinement stage ensures seamless boundaries between the observed and synthesized regions. Second, our framework is training-free and compatible with different 3D foundation models. Third, we introduce Omni-Comp, a comprehensive benchmark combining real-world and synthetic data with diverse and challenging partial patterns, enabling a more thorough and realistic evaluation. Both quantitative and qualitative experiments demonstrate that our approach outperforms previous state-of-the-art approaches. Our code and data will be available at \href{https://github.com/DavidYan2001/LaS-Comp}{LaS-Comp}.

Figures (8)

• Figure 1: Our new framework supports category-agnostic shape completion across diverse partial patterns, including (a) random crops, (b) single-view scans, and (c) missing semantic parts. It further supports both unconditional and text-guided completion, offering flexible control for real-world applications, see (d).
• Figure 2: Overview of the LaS-Comp framework. Starting from Gaussian noise, the process iteratively refines a latent feature $\boldsymbol{x}_t$ under the guidance of the partial input $\boldsymbol{S}_p$. At each iteration $t$, this refinement is performed in two stages: the Explicit Replacement Stage (ERS) and the Implicit Alignment Stage (IAS). The ERS explicitly injects the known geometry of $\boldsymbol{S}_p$ into $\boldsymbol{x}_t$ to produce an updated latent $\boldsymbol{x}^{*}_t$. The IAS then refines $\boldsymbol{x}^{*}_t$ using a gradient-based optimization, yielding a spatially aligned latent $\boldsymbol{x}_{t-dt}$ for the next step. After the final iteration, the completed shape $\boldsymbol{S_{\text{c}}}$ is obtained by decoding the refined latent.
• Figure 3: Overview of the Explicit Replacement Stage (ERS). At each timestep $t$, ERS decomposes the latent generation into two parallel branches. The clean branch (top) enforces spatial consistency, yielding $\boldsymbol{x}^{*}_{0|t}$. Concurrently, the noisy branch (bottom) enhances fidelity, producing $\boldsymbol{x}^{*}_{1|t}$. These two branch outputs are then interpolated to compute the final aligned latent $\boldsymbol{x}^{*}_t$.
• Figure 4: Qualitative comparison on Redwood dataset redwood. We compare with various supervised and unsupervised methods zhu2023svdformeradapointrsdscompletepcdreamercompc, and visualize the output as meshes utilizing the commonly-used mesh reconstruction method Peng2021SAP.
• Figure 5: Visual examples on ScanNet dataset_scannet and KITTI dataset_kitti real-world datasets, which only contain real scans of table, chair, and car, with very sparse points.