Arbitrary-Scale Point Cloud Upsampling by Voxel-Based Network with Latent Geometric-Consistent Learning

TL;DR

This work tackles arbitrary-scale upsampling of sparse, noisy point clouds by introducing PU-VoxelNet, a voxel-based network that treats surface patches as density distributions within regular grid cells. It couples density-guided grid resampling (D-FPS) with a latent geometric-consistent learning objective to improve local surface fidelity, and leverages multi-scale voxelization with a 3D CNN decoder and a refinement module (2D offsets + Point Transformer) to generate high-fidelity points at any rate $r$. The method is trained with a joint loss combining Chamfer-based reconstruction, voxel occupancy/density supervision, a latent-space geometric-consistency term, and regularization, and is shown to outperform state-of-the-art methods on synthetic and real datasets in both fixed and arbitrary-scale upsampling tasks. The results also indicate improved performance in downstream surface reconstruction, and ablations confirm the efficacy of both density-guided resampling and latent geometric-consistent learning for robust surface approximation and detail preservation.

Abstract

Recently, arbitrary-scale point cloud upsampling mechanism became increasingly popular due to its efficiency and convenience for practical applications. To achieve this, most previous approaches formulate it as a problem of surface approximation and employ point-based networks to learn surface representations. However, learning surfaces from sparse point clouds is more challenging, and thus they often suffer from the low-fidelity geometry approximation. To address it, we propose an arbitrary-scale Point cloud Upsampling framework using Voxel-based Network (\textbf{PU-VoxelNet}). Thanks to the completeness and regularity inherited from the voxel representation, voxel-based networks are capable of providing predefined grid space to approximate 3D surface, and an arbitrary number of points can be reconstructed according to the predicted density distribution within each grid cell. However, we investigate the inaccurate grid sampling caused by imprecise density predictions. To address this issue, a density-guided grid resampling method is developed to generate high-fidelity points while effectively avoiding sampling outliers. Further, to improve the fine-grained details, we present an auxiliary training supervision to enforce the latent geometric consistency among local surface patches. Extensive experiments indicate the proposed approach outperforms the state-of-the-art approaches not only in terms of fixed upsampling rates but also for arbitrary-scale upsampling.

Figures (13)

• Figure 1: Differences between previous representative methods Qian2021DeepMUSelfPCU and our voxel-based surface approximation for arbitrary-scale point cloud upsampling. Resorting to the regular structure of voxel grids, the surface patch is approximated as a density distribution of points within each grid cell, and then we can reconstruct an arbitrary number of points from the density predictions.
• Figure 2: (a) Overview of the proposed PU-VoxelNet. Given an input point set, we first convert it to multi-resolution voxels, and then aggregate the multi-scale voxel representations through a 3D CNN based decoder. Next, the grid cells with the desired output number are sampled by a density-guided resampling strategy. Finally, we reconstruct point coordinates from the sampled grid cells. (b) Density-guided grid resampling is developed to sample more faithful grid cells while avoiding sampling outliers. (c) Latent geometric-consistent learning focuses on improving the geometry approximation of local surface patches.
• Figure 3: Analysis of the inaccurate sampling problem on PU1K dataset. Note that we regard the adjacent grid cells as right predictions in this experiment. (a) Percentage of the ground-truth points in sampled cells. (b) Chamfer Distance between sampled cells and the ground truth. (c) Percentage of the ground-truth cells missed by sampling. Previous methods Lim2019ACDwang2021voxel (blue and green bar) suffer inferior sampling, i.e., low precision and high missing rate, while our method (orange bars) can sample faithful grid cells which are more close to the underlying surface. (d) Errors with increasing resampling rates. The proposed D-FPS effectively avoids sampling outliers, resulting in stable improvements.
• Figure 4: A toy example shows the differences between Chamfer Distance (CD) and our method. CD only penalizes on point-wise errors, which fails to exploit the relationships within neighbor points, and cannot guarantee the local structure well. In contrast, we constrain the "edge" information around a seed point, which is complementary to improve fine-structured locality of surface patches.
• Figure 5: Upsampling ($4\times$) results on PU1K dataset with input size of 2,048. The points are colored by the nearest distance between the ground truth and the upsampled points. The blue denotes the small errors. One can zoom in the figure for details.