RefComp: A Reference-guided Unified Framework for Unpaired Point Cloud Completion

TL;DR

This paper proposes a novel unpaired point cloud completion framework, namely the Reference-guided Completion (RefComp) framework, which attains strong performance in both the class-aware and class-agnostic training settings and transforms the unpaired completion problem into a shape translation problem, which is solved in the latent feature space of the partial point clouds.

Abstract

The unpaired point cloud completion task aims to complete a partial point cloud by using models trained with no ground truth. Existing unpaired point cloud completion methods are class-aware, i.e., a separate model is needed for each object class. Since they have limited generalization capabilities, these methods perform poorly in real-world scenarios when confronted with a wide range of point clouds of generic 3D objects. In this paper, we propose a novel unpaired point cloud completion framework, namely the Reference-guided Completion (RefComp) framework, which attains strong performance in both the class-aware and class-agnostic training settings. The RefComp framework transforms the unpaired completion problem into a shape translation problem, which is solved in the latent feature space of the partial point clouds. To this end, we introduce the use of partial-complete point cloud pairs, which are retrieved by using the partial point cloud to be completed as a template. These point cloud pairs are used as reference data to guide the completion process. Our RefComp framework uses a reference branch and a target branch with shared parameters for shape fusion and shape translation via a Latent Shape Fusion Module (LSFM) to enhance the structural features along the completion pipeline. Extensive experiments demonstrate that the RefComp framework achieves not only state-of-the-art performance in the class-aware training setting but also competitive results in the class-agnostic training setting on both virtual scans and real-world datasets.

Figures (7)

• Figure 1: Unpaired point cloud completion by shape translation.$c_y$ is corrupted to create $p_y$ by using $p_x$ as the corruption template. $z_{p_y}$ and $z_{p_x}$ are mapped into features representing complete point clouds by the Latent Shape Fusion Module (LSFM), $T$. $z_{c_{\hat{y}}}$ is forced to be in the $Z_\mathcal{C}$ space by a Wasserstein Distance loss, which pulls$z_{c_{\hat{x}}}$ into the complete latent space. Finally, the completed features are decoded back to the complete point clouds $c_{\hat{x}}$ and $c_{\hat{y}}$.
• Figure 2: Our RefComp framework. The reference branch shares the parameters with the target branch to assist in the unpaired point cloud completion. $p_x, p_y$ are encoded by the same encoder $E_p$. Completion is achieved by the Latent Shape Fusion Module (LSFM) with the mask point cloud
• Figure 3: Generation of the reference data. For each point in $p_x$, the degradation module uses K-nearest neighbour (KNN) to find the top K closest points in $c_y$, where $p_y$ is the union of these points. A CD loss between each $p_y$ and $p_x$ is used to find the top-N reference pairs.
• Figure 4: The Latent Shape Fusion Module (LSFM). The parameters between the reference and the target branch are shared to compute $z_{\hat{c}_x}$ and $z_{\hat{c}_y}$. To complete the partial point clouds in the latent feature space, the LSFM fuses the features of the partial point clouds, i.e., $z_{p_x}$ and $z_{p_y}$, with $z_{m_y}$, which represents missing structural features
• Figure 5: Sample completion results on the CRN dataset. All versions of our framework produce plausible results. The version RefComp w/dis further enhances edges. The reference data used by the three versions of RefComp is shown in yellow.