Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

3D Shape Completion on Unseen Categories:A Weakly-supervised Approach

Lintai Wu, Junhui Hou, Linqi Song, Yong Xu

TL;DR

This work tackles 3D shape completion for unseen categories by introducing a weakly-supervised two-stage framework. It first learns a coarse shape with a Prior-assisted Shape Learning Network (PSLN) that leverages a prior bank from seen categories and a Multi-scale Pattern Correlation module, then refines the result through Category-specific Shape Refinement using voxel-based partial matching losses and category-specific priors. The approach combines supervised-like learning on seen data with self-supervised refinement on unseen data, yielding substantial improvements over state-of-the-art methods on ShapeNet and ScanNet, while also reducing computational demands relative to prior patch-based methods. The results demonstrate robust performance gains, improved efficiency, and a viable path for generalizing 3D shape completion across category boundaries in real-world applications.

Abstract

3D shapes captured by scanning devices are often incomplete due to occlusion. 3D shape completion methods have been explored to tackle this limitation. However, most of these methods are only trained and tested on a subset of categories, resulting in poor generalization to unseen categories. In this paper, we introduce a novel weakly-supervised framework to reconstruct the complete shapes from unseen categories. We first propose an end-to-end prior-assisted shape learning network that leverages data from the seen categories to infer a coarse shape. Specifically, we construct a prior bank consisting of representative shapes from the seen categories. Then, we design a multi-scale pattern correlation module for learning the complete shape of the input by analyzing the correlation between local patterns within the input and the priors at various scales. In addition, we propose a self-supervised shape refinement model to further refine the coarse shape. Considering the shape variability of 3D objects across categories, we construct a category-specific prior bank to facilitate shape refinement. Then, we devise a voxel-based partial matching loss and leverage the partial scans to drive the refinement process. Extensive experimental results show that our approach is superior to state-of-the-art methods by a large margin.

3D Shape Completion on Unseen Categories:A Weakly-supervised Approach

TL;DR

This work tackles 3D shape completion for unseen categories by introducing a weakly-supervised two-stage framework. It first learns a coarse shape with a Prior-assisted Shape Learning Network (PSLN) that leverages a prior bank from seen categories and a Multi-scale Pattern Correlation module, then refines the result through Category-specific Shape Refinement using voxel-based partial matching losses and category-specific priors. The approach combines supervised-like learning on seen data with self-supervised refinement on unseen data, yielding substantial improvements over state-of-the-art methods on ShapeNet and ScanNet, while also reducing computational demands relative to prior patch-based methods. The results demonstrate robust performance gains, improved efficiency, and a viable path for generalizing 3D shape completion across category boundaries in real-world applications.

Abstract

3D shapes captured by scanning devices are often incomplete due to occlusion. 3D shape completion methods have been explored to tackle this limitation. However, most of these methods are only trained and tested on a subset of categories, resulting in poor generalization to unseen categories. In this paper, we introduce a novel weakly-supervised framework to reconstruct the complete shapes from unseen categories. We first propose an end-to-end prior-assisted shape learning network that leverages data from the seen categories to infer a coarse shape. Specifically, we construct a prior bank consisting of representative shapes from the seen categories. Then, we design a multi-scale pattern correlation module for learning the complete shape of the input by analyzing the correlation between local patterns within the input and the priors at various scales. In addition, we propose a self-supervised shape refinement model to further refine the coarse shape. Considering the shape variability of 3D objects across categories, we construct a category-specific prior bank to facilitate shape refinement. Then, we devise a voxel-based partial matching loss and leverage the partial scans to drive the refinement process. Extensive experimental results show that our approach is superior to state-of-the-art methods by a large margin.
Paper Structure (24 sections, 8 equations, 10 figures, 10 tables)

This paper contains 24 sections, 8 equations, 10 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Supervised 3D Shape Completion and Reconstruction
  4. Unsupervised 3D Shape Completion and Reconstruction
  5. Proposed Method
  6. Prior-assisted Coarse Shape Learning
  7. Category-specific Shape Refinement
  8. Experiments
  9. Dataset and Evaluation Metric
  10. Dataset
  11. Evaluation Metric
  12. Implementation Details
  13. Comparison with State-of-the-Art Methods
  14. Results on synthetic data
  15. Results on real-world data
  16. ...and 9 more sections

Figures (10)

  • Figure 1: Flowchart of the Prior-assisted Shape Learning Network. "Conv" and "De-Conv" denote the convolution and de-convolution operation, respectively. The PSLN leverages a Multi-scale Pattern Correlation (MPC) module to capture the associated patterns for each local region of the partial input from the priors and subsequently use these patterns to infer the complete shape hierarchically. The MPC consists of two encoders and three Cross-Attention modules. The prior encoder utilizes multiple kernels of varying sizes to capture local structures at different scales. The Cross-Attention module identifies associated local patterns by calculating attention scores between the multi-scale local features of the partial input and the priors.
  • Figure 2: Visualization of some category-specific priors. These priors contain typical structures specific to their respective categories. (a) bag, (b) basket, (c) bench, (d) printer.
  • Figure 3: Comparison of the visual results on the ShapeNet dataset. From top to bottom, the categories of the objects are bag, lamp, bathtub, bed, basket, printer, laptop, and bench. (a) Partial input (b) AutoSDF. (c) IFNet. (d) Fewshot. (e) SDFusion. (f) PatchComplete. (g) Ours. (h) Ground truth.
  • Figure 4: Comparison of the visual results on ScanNet dataset. From top to bottom, the categories of the objects are bag, lamp, bathtub, bed, basket, and printer. (a) Partial input (b) AutoSDF. (c) IFNet. (d) Fewshot. (e) SDFusion. (f) PatchComplete. (g) Ours. (h) Ground truth.
  • Figure 5: Comparison of visual results between our methods and point cloud completion methods. (a) Partial input, (b) GRNet xie2020grnet, (c) AnchorFormer chen2023anchorformer, (d) Ours, (e) Ground truth.
  • ...and 5 more figures