Table of Contents
Fetching ...

Deep Learning Based 3D Segmentation: A Survey

Yong He, Hongshan Yu, Xiaoyan Liu, Zhengeng Yang, Wei Sun, Saeed Anwar, Ajmal Mian

TL;DR

This paper comprehensively surveys the recent progress in deep learning-based 3D segmentation techniques, covering over 220 works from the last six years, analyze their strengths and limitations, and discusses their competitive results on benchmark datasets.

Abstract

3D segmentation is a fundamental and challenging problem in computer vision with applications in autonomous driving and robotics. It has received significant attention from the computer vision, graphics and machine learning communities. Conventional methods for 3D segmentation, based on hand-crafted features and machine learning classifiers, lack generalization ability. Driven by their success in 2D computer vision, deep learning techniques have recently become the tool of choice for 3D segmentation tasks. This has led to an influx of many methods in the literature that have been evaluated on different benchmark datasets. Whereas survey papers on RGB-D and point cloud segmentation exist, there is a lack of a recent in-depth survey that covers all 3D data modalities and application domains. This paper fills the gap and comprehensively surveys the recent progress in deep learning-based 3D segmentation techniques. We cover over 220 works from the last six years, analyze their strengths and limitations, and discuss their competitive results on benchmark datasets. The survey provides a summary of the most commonly used pipelines and finally highlights promising research directions for the future.

Deep Learning Based 3D Segmentation: A Survey

TL;DR

This paper comprehensively surveys the recent progress in deep learning-based 3D segmentation techniques, covering over 220 works from the last six years, analyze their strengths and limitations, and discusses their competitive results on benchmark datasets.

Abstract

3D segmentation is a fundamental and challenging problem in computer vision with applications in autonomous driving and robotics. It has received significant attention from the computer vision, graphics and machine learning communities. Conventional methods for 3D segmentation, based on hand-crafted features and machine learning classifiers, lack generalization ability. Driven by their success in 2D computer vision, deep learning techniques have recently become the tool of choice for 3D segmentation tasks. This has led to an influx of many methods in the literature that have been evaluated on different benchmark datasets. Whereas survey papers on RGB-D and point cloud segmentation exist, there is a lack of a recent in-depth survey that covers all 3D data modalities and application domains. This paper fills the gap and comprehensively surveys the recent progress in deep learning-based 3D segmentation techniques. We cover over 220 works from the last six years, analyze their strengths and limitations, and discuss their competitive results on benchmark datasets. The survey provides a summary of the most commonly used pipelines and finally highlights promising research directions for the future.

Paper Structure

This paper contains 27 sections, 7 equations, 9 figures, 10 tables.

Figures (9)

  • Figure 1: The main three types of 3D segmentation are (b) 3D semantic segmentation, (c) 3D instance segmentation, and (d) 3D part segmentation on (a) 3D point clouds.
  • Figure 2: Complete overview of the survey paper. Highlighting each section and its essential content.
  • Figure 3: The four types of 3D data are (a) RGBD, (b) Point cloud, (c) Voxel, and (d) Mesh.
  • Figure 4: Annotated examples from (a) S3DIS, (b) Semantic3D, (c) SemanticKITTI for 3D semantic segmentation, (d) ScanNet for 3D instance segmentation, and (e) ShapeNet for 3D part segmentation. See Table \ref{['table1']} for a summary of these datasets.
  • Figure 5: Milestones of deep learning based 3D semantic segmentation methods. Note that the arrow (timeline) goes anti-clockwise
  • ...and 4 more figures