PLATYPUS: Progressive Local Surface Estimator for Arbitrary-Scale Point Cloud Upsampling

TL;DR

This work addresses point cloud upsampling challenges by introducing the Progressive Local Surface Estimator (PLSE), which more effectively captures local features in complex regions through a curvature-based sampling technique that selectively targets high-curvature areas.

Abstract

3D point clouds are increasingly vital for applications like autonomous driving and robotics, yet the raw data captured by sensors often suffer from noise and sparsity, creating challenges for downstream tasks. Consequently, point cloud upsampling becomes essential for improving density and uniformity, with recent approaches showing promise by projecting randomly generated query points onto the underlying surface of sparse point clouds. However, these methods often result in outliers, non-uniformity, and difficulties in handling regions with high curvature and intricate structures. In this work, we address these challenges by introducing the Progressive Local Surface Estimator (PLSE), which more effectively captures local features in complex regions through a curvature-based sampling technique that selectively targets high-curvature areas. Additionally, we incorporate a curriculum learning strategy that leverages the curvature distribution within the point cloud to naturally assess the sample difficulty, enabling curriculum learning on point cloud data for the first time. The experimental results demonstrate that our approach significantly outperforms existing methods, achieving high-quality, dense point clouds with superior accuracy and detail.

Figures (11)

• Figure 1: Comparative visualization of 4$\times$ point cloud upsampling results on PU1K. Point cloud upsampling is the task of generating a denser point cloud (i.e., rightmost column) that accurately reflects the underlying geometry of a sparse point cloud (i.e., leftmost column). Our method successfully upsamples intricate areas where existing methods struggle to perform well.
• Figure 2: An analysis of the differences between FPS (Farthest Point Sampling) and our newly proposed curvature-based sampling. The comparison shows which points remain as the point cloud is progressively sampled down to fewer points using each sampling method. FPS uniformly samples points across the entire point cloud, whereas curvature-based sampling selectively samples points from regions with intricate structures and high curvature values.
• Figure 3: An analysis of the easy samples and hard samples used in our curriculum learning strategy. (a) Point clouds with a higher proportion of points with low curvature values, resulting in a distribution with high skewness, were classified as easy samples. (b) Conversely, point clouds with a higher proportion of points with high curvature values were classified as hard samples.
• Figure 4: Overall Pipeline of PLATYPUS. Input: The input consists of a sparse point cloud $\mathbf{P_{input}}$ and nearby generated query points $\mathbf{q}$. Train: During the training process, our Progressive Local Surface Estimator (PLSE) $g_\theta$ is trained to predict the distance from the query point to the underlying surface of the sparse point cloud. The loss compares the distance from the query point $\mathbf{q}$ to the surface of the sparse point cloud (assumed to be identical to the ground truth point cloud). Inference: the initial query point $\mathbf{q}^0$ is progressively updates using the PLSE gradient $\nabla g_\theta$ to become the final query $\mathbf{q}^T$ projected onto the surface of the input point cloud.
• Figure 5: Visualization of each point's curvature value. Points with high curvature values are shown in red, while points with low curvature values are shown in blue.