Digging into Intrinsic Contextual Information for High-fidelity 3D Point Cloud Completion

TL;DR

The paper addresses occlusion-induced incompleteness in 3D point clouds and proposes a two-stage completion framework comprising a diffusion-based Coarse Generator (DCG) and a Context-aware Refiner (CRef). The refinement integrates short-range contextual information via a mixed sampling and surface freezing and long-range contextual information through a learnable rigid-transformation invariant similarity module plus non-local refinement. Key contributions include the combination of short-range and long-range contextual extraction with a non-local similarity-based refinement network, achieving state-of-the-art results on ShapeNet-ViPC and other benchmarks. The approach offers practical gains in high-fidelity PCC and is accompanied by public release of code.

Abstract

The common occurrence of occlusion-induced incompleteness in point clouds has made point cloud completion (PCC) a highly-concerned task in the field of geometric processing. Existing PCC methods typically produce complete point clouds from partial point clouds in a coarse-to-fine paradigm, with the coarse stage generating entire shapes and the fine stage improving texture details. Though diffusion models have demonstrated effectiveness in the coarse stage, the fine stage still faces challenges in producing high-fidelity results due to the ill-posed nature of PCC. The intrinsic contextual information for texture details in partial point clouds is the key to solving the challenge. In this paper, we propose a high-fidelity PCC method that digs into both short and long-range contextual information from the partial point cloud in the fine stage. Specifically, after generating the coarse point cloud via a diffusion-based coarse generator, a mixed sampling module introduces short-range contextual information from partial point clouds into the fine stage. A surface freezing modules safeguards points from noise-free partial point clouds against disruption. As for the long-range contextual information, we design a similarity modeling module to derive similarity with rigid transformation invariance between points, conducting effective matching of geometric manifold features globally. In this way, the high-quality components present in the partial point cloud serve as valuable references for refining the coarse point cloud with high fidelity. Extensive experiments have demonstrated the superiority of the proposed method over SOTA competitors. Our code is available at https://github.com/JS-CHU/ContextualCompletion.

Figures (5)

• Figure 1: The workflow of the proposed method. In the middle of the figure, we visualize the matching of non-local regions based on the similarity of geometric structures. A higher degree of similarity is represented by a more intense yellow color. The green point in the sampled point cloud is refined refering to the heatmap in the partial point cloud.
• Figure 2: The overall architecture consists of a Diffusion-based Coarse Generator (DCG) and a Context-aware Refiner (CRef). In DCG, a PointNet Encoder extracts a global shape latent code from the partial input as a condition. The coarse point cloud is generated through denoising process. The coarse point cloud is then refined in CRef according to both short and long-range contextual information, deriving a point cloud with entire shape and high-fidelity textural details.
• Figure 3: A patch is firstly rotated to a certain direction by $\theta$ and secondly rotated by the learned angle $\phi$. Finally, the patch performs symmetry with respect to the symmetry plane defined by the learned angle $\psi$.
• Figure 4: Qualitative comparison on ShapeNet-ViPC. The resolution for all point clouds are 2,048.
• Figure 5: For each pair of images, the left image highlights a specific point within the complete point cloud. The accompanying heatmap on the right displays the similarity of each point in the partial point cloud to that reference point. A higher degree of similarity is indicated by more intense colors: red for airplanes and yellow for chairs and cars.