Deep Learning for 3D Point Cloud Enhancement: A Survey

TL;DR

This survey presents a new taxonomy for recent state-of-the-art methods and systematic experimental results on standard benchmarks, and shares the insightful observations, thoughts, and inspiring future research directions for point cloud enhancement with deep learning.

Abstract

Point cloud data now are popular data representations in a number of three-dimensional (3D) vision research realms. However, due to the limited performance of sensors and sensing noise, the raw data usually suffer from sparsity, noise, and incompleteness. This poses great challenges to down-stream point cloud processing tasks. In recent years, deep-learning-based point cloud enhancement methods, which aim to achieve dense, clean, and complete point clouds from low-quality raw point clouds using deep neural networks, are gaining tremendous research attention. This paper, for the first time to our knowledge, presents a comprehensive survey for deep-learning-based point cloud enhancement methods. It covers three main perspectives for point cloud enhancement, i.e., (1) denoising to achieve clean data; (2) completion to recover unseen data; (3) upsampling to obtain dense data. Our survey presents a new taxonomy for recent state-of-the-art methods and systematic experimental results on standard benchmarks. In addition, we share our insightful observations, thoughts, and inspiring future research directions for point cloud enhancement with deep learning.

Figures (21)

• Figure 1: Illustration of 3D point cloud enhancement. It aims to generate smooth, complete, and dense point clouds from low-quality raw point clouds. In point cloud enhancement, point cloud denoising, completion, and upsampling are three main tasks. Specifically, denosing aims to eliminate or rectify noisy points; completion is to recover the missing part in point clouds; upsampling is to enhance the resolution of the point cloud.
• Figure 2: A taxonomy of 3D point cloud enhancement.
• Figure 3: A detailed taxonomy of point cloud denoising methods.
• Figure 4: Chronological overview of representative point cloud denoising networks.
• Figure 5: Illustration of the point-based methods. (a) Displacement-based methods. (b) Surface-based methods. (c) Gradient-based methods.