Joint Point Cloud Upsampling and Cleaning with Octree-based CNNs

TL;DR

This work embraces simplicity and presents a simple yet efficient method for jointly upsampling and cleaning point clouds, leveraging an off-the-shelf octree-based 3D U-Net with minor modifications, enabling the upsampling and cleaning tasks within a single network.

Abstract

Recovering dense and uniformly distributed point clouds from sparse or noisy data remains a significant challenge. Recently, great progress has been made on these tasks, but usually at the cost of increasingly intricate modules or complicated network architectures, leading to long inference time and huge resource consumption. Instead, we embrace simplicity and present a simple yet efficient method for jointly upsampling and cleaning point clouds. Our method leverages an off-the-shelf octree-based 3D U-Net (OUNet) with minor modifications, enabling the upsampling and cleaning tasks within a single network. Our network directly processes each input point cloud as a whole instead of processing each point cloud patch as in previous works, which significantly eases the implementation and brings at least 47 times faster inference. Extensive experiments demonstrate that our method achieves state-of-the-art performances under huge efficiency advantages on a series of benchmarks. We expect our method to serve simple baselines and inspire researchers to rethink the method design on point cloud upsampling and cleaning.

Figures (9)

• Figure 1: Comparisons of our method (OUNet) with other state-of-the-art methods. The horizontal and vertical axes represent the running time of generating an entire point cloud and the Chamfer distance, respectively. (a): The results of upsampling on the PU-GAN dataset from 5k to 80k points. (b): The results of cleaning 10k points with noise level 0.02. (c): The results of cleaning 50k points with noise level 0.02. Our method achieves the best Chamfer distances while running the fastest. It runs at least 47 times faster than the others on point upsampling.
• Figure 2: Pipeline comparisons between our OUNet and other methods on point cloud upsampling and cleaning. (a): OUNet. The model consists of multiple octree-based Residual Blocks, Downsampling, and Upsampling components. (b)-(d): These 3 methods all require splitting point cloud into patches using FPS and K-Nearest Neighbors (K-NN) and upsampling or cleaning them, followed by stitching patches. In this figure, all icons sharing the same shape and color convey identical meanings. The yellow and orange rectangles denote input and output point cloud, respectively.
• Figure 3: Architecture of OUNet. We represent the input point cloud as octrees. The architecture is simple, consisting of the encoder, decoder and skip connections. Both the encoder and decoder employ stacked octree-based residual blocks.
• Figure 4: Qualitative upsampling results on the PU-GAN dataset. We provide a zoom-in view of the orange box for clearer observation. Outputs of our model are visually better, with fewer outliers, more fine-grained and sharp geometry and more uniform distribution.
• Figure 5: Qualitative upsampling results on the Sketchfab dataset.