Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Multimodal Point Cloud Completion: A Completion-by-Correction Perspective

Wang Luo, Di Wu, Hengyuan Na, Yinlin Zhu, Miao Hu, Guocong Quan

TL;DR

This work challenges the dominant Completion-by-Inpainting approach for multimodal point cloud completion by proposing a Completion-by-Correction paradigm that grounds a complete image-to-3D prior in the partial observation. It introduces PGNet, a three-stage framework with corrective dual-feature encoding, grounded seed generation, and hierarchical refinement to produce observation-aligned, structurally coherent completions. On ShapeNet-ViPC, PGNet achieves state-of-the-art performance, significantly reducing Chamfer Distance and boosting F-score, while ablations confirm the critical role of dual-source grounding and structure-aware refinement. The approach offers a robust, scalable path for reliable 3D reconstruction from partial data in cluttered or occluded environments.

Abstract

Point cloud completion aims to reconstruct complete 3D shapes from partial observations, which is a challenging problem due to severe occlusions and missing geometry. Despite recent advances in multimodal techniques that leverage complementary RGB images to compensate for missing geometry, most methods still follow a Completion-by-Inpainting paradigm, synthesizing missing structures from fused latent features. We empirically show that this paradigm often results in structural inconsistencies and topological artifacts due to limited geometric and semantic constraints. To address this, we rethink the task and propose a more robust paradigm, termed Completion-by-Correction, which begins with a topologically complete shape prior generated by a pretrained image-to-3D model and performs feature-space correction to align it with the partial observation. This paradigm shifts completion from unconstrained synthesis to guided refinement, enabling structurally consistent and observation-aligned reconstruction. Building upon this paradigm, we introduce PGNet, a multi-stage framework that conducts dual-feature encoding to ground the generative prior, synthesizes a coarse yet structurally aligned scaffold, and progressively refines geometric details via hierarchical correction. Experiments on the ShapeNetViPC dataset demonstrate the superiority of PGNet over state-of-the-art baselines in terms of average Chamfer Distance (-23.5%) and F-score (+7.1%).

Rethinking Multimodal Point Cloud Completion: A Completion-by-Correction Perspective

TL;DR

This work challenges the dominant Completion-by-Inpainting approach for multimodal point cloud completion by proposing a Completion-by-Correction paradigm that grounds a complete image-to-3D prior in the partial observation. It introduces PGNet, a three-stage framework with corrective dual-feature encoding, grounded seed generation, and hierarchical refinement to produce observation-aligned, structurally coherent completions. On ShapeNet-ViPC, PGNet achieves state-of-the-art performance, significantly reducing Chamfer Distance and boosting F-score, while ablations confirm the critical role of dual-source grounding and structure-aware refinement. The approach offers a robust, scalable path for reliable 3D reconstruction from partial data in cluttered or occluded environments.

Abstract

Point cloud completion aims to reconstruct complete 3D shapes from partial observations, which is a challenging problem due to severe occlusions and missing geometry. Despite recent advances in multimodal techniques that leverage complementary RGB images to compensate for missing geometry, most methods still follow a Completion-by-Inpainting paradigm, synthesizing missing structures from fused latent features. We empirically show that this paradigm often results in structural inconsistencies and topological artifacts due to limited geometric and semantic constraints. To address this, we rethink the task and propose a more robust paradigm, termed Completion-by-Correction, which begins with a topologically complete shape prior generated by a pretrained image-to-3D model and performs feature-space correction to align it with the partial observation. This paradigm shifts completion from unconstrained synthesis to guided refinement, enabling structurally consistent and observation-aligned reconstruction. Building upon this paradigm, we introduce PGNet, a multi-stage framework that conducts dual-feature encoding to ground the generative prior, synthesizes a coarse yet structurally aligned scaffold, and progressively refines geometric details via hierarchical correction. Experiments on the ShapeNetViPC dataset demonstrate the superiority of PGNet over state-of-the-art baselines in terms of average Chamfer Distance (-23.5%) and F-score (+7.1%).

Paper Structure

This paper contains 24 sections, 14 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Unimodal Point Cloud Completion
  4. Multimodal Point Cloud Completion
  5. Methodology
  6. Overview
  7. Problem Definition.
  8. The Completion-by-Correction Paradigm.
  9. Corrective Dual-Feature Encoding
  10. Partial Point Cloud Encoder.
  11. Generative Point Cloud Encoder.
  12. Grounded Seed Generation
  13. Hierarchical Grounded Refinement
  14. Dual-Source Feature Association.
  15. Structure-Aware Upsampling.
  16. ...and 9 more sections

Figures (3)

  • Figure 1: Comparison of two point cloud completion paradigms. (a) Completion-by-Inpainting synthesizes missing geometry from incomplete representation, often introducing artifacts. (b) Our Completion-by-Correction leverages a complete generative prior, correcting it by grounding to the partial observation for more consistent structures.
  • Figure 2: The overall architecture of PriorGroundNet (PGNet), which follows the Completion-by-Correction paradigm and consists of three stages: Corrective Dual-Feature Encoding, Grounded Seed Generation, and Hierarchical Grounded Refinement. (a) The detailed structure of the Seed Generator module, which produces a coarse but complete point cloud by grounding semantic seeds. (b) The architecture of the Grounded Refinement Block (GRB), which hierarchically enhances geometric detail using dual-source feature.
  • Figure 3: Qualitative comparison on ShapeNet-ViPC.