REPS: Reconstruction-based Point Cloud Sampling

TL;DR

Efficient downsampling of 3D point clouds is essential for scalable 3D vision. The paper proposes REPS, a reconstruction-based sampling framework that evaluates point importance via point reconstruction and shape reconstruction losses, guided by the GLFA module to balance local and global information. Downsampling selects the top-$M$ points according to a total score, and the model is trained with $\mathcal{L}_{total} = \mathcal{L}_{sample} + \mathcal{L}_{task}$ to optimize both reconstruction quality and downstream task performance. Experiments on ModelNet40 and ShapeNetPart demonstrate superior preservation of geometric structure and improved downstream accuracy, validating the approach across classification and segmentation tasks.

Abstract

Sampling is widely used in various point cloud tasks as it can effectively reduce resource consumption. Recently, some methods have proposed utilizing neural networks to optimize the sampling process for various task requirements. Currently, deep downsampling methods can be categorized into two main types: generative-based and score-based. Generative-based methods directly generate sampled point clouds using networks, whereas score-based methods assess the importance of points according to specific rules and then select sampled point clouds based on their scores. However, these methods often result in noticeable clustering effects in high-intensity feature areas, compromising their ability to preserve small-scale features and leading to the loss of some structures, thereby affecting the performance of subsequent tasks. In this paper, we propose REPS, a reconstruction-based scoring strategy that evaluates the importance of each vertex by removing and reconstructing them using surrounding vertices. Our reconstruction process comprises point reconstruction and shape reconstruction. The two aforementioned reconstruction methods effectively evaluate the importance of vertices by removing them at different scales for reconstruction. These reconstructions ensure that our method maintains the overall geometric features of the point cloud and avoids disturbing small-scale structures during sampling. Additionally, we propose the Global-Local Fusion Attention (GLFA) module, which aggregates local and global attention features of point clouds, ensuring high-quality reconstruction and sampling effects. Our method outperforms previous approaches in preserving the structural features of the sampled point clouds. Furthermore, abundant experimental results demonstrate the superior performance of our method across various common tasks.

Figures (7)

• Figure 1: Left: Point Reconstruction. For each point in the input point cloud, we select its K nearest neighbors and then reconstruct the central point based on the features of these neighboring points. Right: Shape Reconstruction. For each local patch, we randomly remove some points and then use the remaining points to reconstruct the complete local shape.
• Figure 2: Top: Visualization of scores based on reconstruction. Bottom: Distribution of scores based on point reconstruction, scores based on shape reconstruction, and total scores.
• Figure 3: The overview of our method. Left: Architecture of semantic segmentation and classification models. Right: Implementation details of the GLFA module.
• Figure 4: Point cloud sampling visualization results on the ModelNet40 dataset. At the top is the original point cloud (1024 points), followed by downsampled point clouds with sampling ratios of 0.5, 0.25, and 0.125, respectively.
• Figure 5: Visualizations of segmentation results on the ShapeNetPart dataset.