Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Imagine with the Teacher: Complete Shape in a Multi-View Distillation Way

Zhanpeng Luo, Linna Wang, Guangwu Qian, Li Lu

TL;DR

VD-PCN addresses the challenge of completing 3D point clouds from partial observations by projecting the input into depth images from multiple viewpoints and processing them with a shared 2D encoder. It introduces a teacher–student knowledge distillation framework to transfer rich ground-truth information into the completion pipeline, guided by a feature-level alignment between teacher and student. A dual-modality decoder integrates 2D and 3D cues to upsample coarse predictions to high-resolution completed shapes. Across PCN, ShapeNet-55, and MVP benchmarks, VD-PCN achieves state-of-the-art accuracy with favorable compute-efficiency trade-offs, demonstrating the practicality of combining multi-view perception with distillation for 3D completion.

Abstract

Point cloud completion aims to recover the completed 3D shape of an object from its partial observation caused by occlusion, sensor's limitation, noise, etc. When some key semantic information is lost in the incomplete point cloud, the neural network needs to infer the missing part based on the input information. Intuitively we would apply an autoencoder architecture to solve this kind of problem, which take the incomplete point cloud as input and is supervised by the ground truth. This process that develops model's imagination from incomplete shape to complete shape is done automatically in the latent space. But the knowledge for mapping from incomplete to complete still remains dark and could be further explored. Motivated by the knowledge distillation's teacher-student learning strategy, we design a knowledge transfer way for completing 3d shape. In this work, we propose a novel View Distillation Point Completion Network (VD-PCN), which solve the completion problem by a multi-view distillation way. The design methodology fully leverages the orderliness of 2d pixels, flexibleness of 2d processing and powerfulness of 2d network. Extensive evaluations on PCN, ShapeNet55/34, and MVP datasets confirm the effectiveness of our design and knowledge transfer strategy, both quantitatively and qualitatively. Committed to facilitate ongoing research, we will make our code publicly available.

Imagine with the Teacher: Complete Shape in a Multi-View Distillation Way

TL;DR

VD-PCN addresses the challenge of completing 3D point clouds from partial observations by projecting the input into depth images from multiple viewpoints and processing them with a shared 2D encoder. It introduces a teacher–student knowledge distillation framework to transfer rich ground-truth information into the completion pipeline, guided by a feature-level alignment between teacher and student. A dual-modality decoder integrates 2D and 3D cues to upsample coarse predictions to high-resolution completed shapes. Across PCN, ShapeNet-55, and MVP benchmarks, VD-PCN achieves state-of-the-art accuracy with favorable compute-efficiency trade-offs, demonstrating the practicality of combining multi-view perception with distillation for 3D completion.

Abstract

Point cloud completion aims to recover the completed 3D shape of an object from its partial observation caused by occlusion, sensor's limitation, noise, etc. When some key semantic information is lost in the incomplete point cloud, the neural network needs to infer the missing part based on the input information. Intuitively we would apply an autoencoder architecture to solve this kind of problem, which take the incomplete point cloud as input and is supervised by the ground truth. This process that develops model's imagination from incomplete shape to complete shape is done automatically in the latent space. But the knowledge for mapping from incomplete to complete still remains dark and could be further explored. Motivated by the knowledge distillation's teacher-student learning strategy, we design a knowledge transfer way for completing 3d shape. In this work, we propose a novel View Distillation Point Completion Network (VD-PCN), which solve the completion problem by a multi-view distillation way. The design methodology fully leverages the orderliness of 2d pixels, flexibleness of 2d processing and powerfulness of 2d network. Extensive evaluations on PCN, ShapeNet55/34, and MVP datasets confirm the effectiveness of our design and knowledge transfer strategy, both quantitatively and qualitatively. Committed to facilitate ongoing research, we will make our code publicly available.

Paper Structure

This paper contains 26 sections, 4 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Point cloud completion
  4. Multi-View 3d Understanding
  5. Knowledge Distillation
  6. Methodology
  7. General Pipeline
  8. Model Design
  9. Multi-View Perceptual Encoder
  10. Distilled Knowledge Transfer
  11. Dual-Modality Decoder
  12. Training Loss
  13. Experiment
  14. Experiment Setting
  15. Dataset
  16. ...and 11 more sections

Figures (3)

  • Figure 1: Model Overview
  • Figure 2: Model detail illustrated for three modules in the decoder part.
  • Figure 3: Visualizartion on the PCN Dataset.