PointCFormer: a Relation-based Progressive Feature Extraction Network for Point Cloud Completion

TL;DR

PointCFormer tackles incomplete 3D point clouds by integrating a transformer-based encoder-decoder with a local geometric relationship perception that uses dual metrics $R_1$ and $R_2$ to weight neighbor contributions, and a progressive feature extractor that alternates global and local queries to preserve global structure while improving local detail. A latent-space point proxy correction module reuses dense input information to enrich generated proxies, improving alignment with the original distribution. The model is trained with a Chamfer-based objective plus a denoising auxiliary loss, achieving state-of-the-art results on PCN, ShapeNet-55, ShapeNet-34/Unseen-21, and KITTI, and demonstrating strong generalization and robustness to sparse inputs. These contributions advance reliable 3D shape completion with improved geometric fidelity, enabling better downstream perception tasks in real-world scenarios.

Abstract

Point cloud completion aims to reconstruct the complete 3D shape from incomplete point clouds, and it is crucial for tasks such as 3D object detection and segmentation. Despite the continuous advances in point cloud analysis techniques, feature extraction methods are still confronted with apparent limitations. The sparse sampling of point clouds, used as inputs in most methods, often results in a certain loss of global structure information. Meanwhile, traditional local feature extraction methods usually struggle to capture the intricate geometric details. To overcome these drawbacks, we introduce PointCFormer, a transformer framework optimized for robust global retention and precise local detail capture in point cloud completion. This framework embraces several key advantages. First, we propose a relation-based local feature extraction method to perceive local delicate geometry characteristics. This approach establishes a fine-grained relationship metric between the target point and its k-nearest neighbors, quantifying each neighboring point's contribution to the target point's local features. Secondly, we introduce a progressive feature extractor that integrates our local feature perception method with self-attention. Starting with a denser sampling of points as input, it iteratively queries long-distance global dependencies and local neighborhood relationships. This extractor maintains enhanced global structure and refined local details, without generating substantial computational overhead. Additionally, we develop a correction module after generating point proxies in the latent space to reintroduce denser information from the input points, enhancing the representation capability of the point proxies. PointCFormer demonstrates state-of-the-art performance on several widely used benchmarks. Our code is available at https://github.com/Zyyyyy0926/PointCFormer_Plus_Pytorch.

Figures (8)

• Figure 1: Improved global shape and enriched local detail in point cloud completion using our method.
• Figure 2: Overview of PointCFormer framework: Initially, we extract representative sampling points and their local features from the input incomplete point cloud using a feature extractor. After adding position embeddings to the local features, we employ a Transformer encoder-decoder architecture to predict point proxies for the missing parts. Concurrently, a correction module aligns these point proxies with the original point cloud distribution. Finally, a simple MLP and a Rebuild head are used to complete the point cloud based on the predicted point proxies in a coarse-to-fine manner.
• Figure 3: Comparison of local feature extraction: traditional kNN-based method (Left); DGCNN with relative relation fusion (Center); Ours (Right). In (c), the redder the line segment, the stronger the correlation between the two connected points; the bluer the line segment, the weaker the correlation.
• Figure 4: Quantification of the relationship between two points.
• Figure 5: The framework of local geometric relationship perception.