CS-Net:Contribution-based Sampling Network for Point Cloud Simplification

TL;DR

The paper tackles efficient, task-aware downsampling of 3D point clouds by formulating selective sampling as a differentiable $Top-k$ operation guided by per-point contributions. It proposes CS-Net, composed of a Feature Embedding module, Cascade Attention, and a Contribution Scoring module, with an entropy-regularized optimal transport formulation enabling end-to-end training and preventing duplicate samples. The method jointly optimizes a transport-based sampling objective and a downstream task loss, achieving state-of-the-art results on ModelNet40, PU147, and KITTI for classification, registration, compression, surface reconstruction, and object detection, while demonstrating favorable time complexity. This approach offers practical benefits for processing large-scale point clouds by preserving shapes and salient details at various sampling ratios, with potential impact on real-time 3D perception workflows.

Abstract

Point cloud sampling plays a crucial role in reducing computation costs and storage requirements for various vision tasks. Traditional sampling methods, such as farthest point sampling, lack task-specific information and, as a result, cannot guarantee optimal performance in specific applications. Learning-based methods train a network to sample the point cloud for the targeted downstream task. However, they do not guarantee that the sampled points are the most relevant ones. Moreover, they may result in duplicate sampled points, which requires completion of the sampled point cloud through post-processing techniques. To address these limitations, we propose a contribution-based sampling network (CS-Net), where the sampling operation is formulated as a Top-k operation. To ensure that the network can be trained in an end-to-end way using gradient descent algorithms, we use a differentiable approximation to the Top-k operation via entropy regularization of an optimal transport problem. Our network consists of a feature embedding module, a cascade attention module, and a contribution scoring module. The feature embedding module includes a specifically designed spatial pooling layer to reduce parameters while preserving important features. The cascade attention module combines the outputs of three skip connected offset attention layers to emphasize the attractive features and suppress less important ones. The contribution scoring module generates a contribution score for each point and guides the sampling process to prioritize the most important ones. Experiments on the ModelNet40 and PU147 showed that CS-Net achieved state-of-the-art performance in two semantic-based downstream tasks (classification and registration) and two reconstruction-based tasks (compression and surface reconstruction).

Figures (14)

• Figure 1: Architecture of CS-Net.We first use an FE module to capture both local and global characteristics. Then, we use a CA module to emphasize the attractive features and suppress less important ones. A CS module is then connected after the CA module to map the features into point-wise scores.These scores represent the importance of each point with respect to the downstream task and the loss function. We sort these scores and select the points with the $k$ highest scores in the sampled point cloud $\mathbf{P}_{sp}$ by introducing the Top-$k$ operation.
• Figure 2: Proposed FE module. The features of each point in $\mathbf{P}_{in}$ are extracted through the FE module.
• Figure 3: Self-attention (switch connected at side a) and offset-attention (switch connected at side b).
• Figure 4: Architecture of the CA module and CS module. The CA module is used to emphasize the attractive features and suppress less important ones. The CS module is then connected after the CA module to map the features into point-wise scores. By introducing the Top-$k$ operation, the points with the $k$ highest scores are sorted and selected.
• Figure 5: Subjective visual comparison of sampled point clouds with different loss functions. Black points represent the original point cloud, and red points represent the sampled point cloud.