FSC: Few-point Shape Completion

TL;DR

This work tackles 3D shape completion from extremely sparse point clouds, showing that even with around $64$ input points a large portion of ground-truth information can be preserved. It introduces Few-point Shape Completion (FSC), a dual-branch encoder that captures extensive and salient features, and a two-stage revision network that adversarially aligns both latent features and decoded geometry before generating a detailed point cloud. The approach achieves state-of-the-art performance across few-point and many-point settings, with demonstrated robustness to unseen categories and real-world data (e.g., KITTI). The combination of dense feature extraction, dynamic point saliency, and iterative revision provides a practical solution for shape completion in applications with sparse sensory data.

Abstract

While previous studies have demonstrated successful 3D object shape completion with a sufficient number of points, they often fail in scenarios when a few points, e.g. tens of points, are observed. Surprisingly, via entropy analysis, we find that even a few points, e.g. 64 points, could retain substantial information to help recover the 3D shape of the object. To address the challenge of shape completion with very sparse point clouds, we then propose Few-point Shape Completion (FSC) model, which contains a novel dual-branch feature extractor for handling extremely sparse inputs, coupled with an extensive branch for maximal point utilization with a saliency branch for dynamic importance assignment. This model is further bolstered by a two-stage revision network that refines both the extracted features and the decoder output, enhancing the detail and authenticity of the completed point cloud. Our experiments demonstrate the feasibility of recovering 3D shapes from a few points. The proposed Few-point Shape Completion (FSC) model outperforms previous methods on both few-point inputs and many-point inputs, and shows good generalizability to different object categories.

Figures (9)

• Figure 1: We use 64 points as input for point cloud completion. The PCN yuan2018pcn result does not result in good general shapes. GRNet xie2020grnet result has unexpected holes due to the lack of dense representations. In comparison, our few-point shape completion method can provide much more reasonable results.
• Figure 2: FSC pipeline. The dual-branch feature extraction network abstracts the input point cloud $X$ as a rough feature vector $f_{\rm{coarse}}$. The two-stage revision network corrects $f_{\rm{coarse}}$ and the coarse point cloud $Y_{\rm{coarse}}$ as $f_{\rm{fine}}$ and $Y_{\rm{fine}}$, respectively. The detail point cloud $Y_{\rm{detail}}$ is generated by using $f_{\rm{fine}}$, $Y_{\rm{fine}}$ and its point features.
• Figure 3: We analyze the FPFH Shannon Entropy of the input point cloud and completion results on ShapeNet. We observe that when the number of input points drops to 64, the input point cloud still contains 45.51% of the amount of information compared to the original 16,384 points (blue line). When the number of points further decreases, the amount of information drops sharply. In the experiment of this paper, we use 64 as the minimum number of points. The orange line shows the information of the point cloud after completion.
• Figure 4: In our comparative analysis on ShapeNet, we varied the input point number from 2048 to 16. We observe that our model (blue line) consistently outperformed previous methods across different input point counts. Notably, when the number of input points was reduced to below 64, there was a significant decline in performance for all evaluated methods.
• Figure 5: We report the completion result grouped by the number of input points on LiDAR scans of the KITTI dataset. As shown in the figure, our method (green line) outperforms previous methods in every number of input point groups. We also observe a sharp error increase when the number of input points is lower than 64.