Semantics-Guided Moving Object Segmentation with 3D LiDAR

TL;DR

A semantics-guided convolutional neural network for moving object segmentation that takes sequential LiDAR range images as inputs and converts the semantic features of adjacent scans into the same coordinate system to fully exploit the cross-scan semantic features.

Abstract

Moving object segmentation (MOS) is a task to distinguish moving objects, e.g., moving vehicles and pedestrians, from the surrounding static environment. The segmentation accuracy of MOS can have an influence on odometry, map construction, and planning tasks. In this paper, we propose a semantics-guided convolutional neural network for moving object segmentation. The network takes sequential LiDAR range images as inputs. Instead of segmenting the moving objects directly, the network conducts single-scan-based semantic segmentation and multiple-scan-based moving object segmentation in turn. The semantic segmentation module provides semantic priors for the MOS module, where we propose an adjacent scan association (ASA) module to convert the semantic features of adjacent scans into the same coordinate system to fully exploit the cross-scan semantic features. Finally, by analyzing the difference between the transformed features, reliable MOS result can be obtained quickly. Experimental results on the SemanticKITTI MOS dataset proves the effectiveness of our work.

Figures (6)

• Figure 1: Range-based LiDAR segmentation methods. (a) Multiple-scan-based LiDAR semantic segmentation method. It uses consecutive range images as inputs and outputs corresponding semantic segmentation results, including moving information. (b) LiDAR-based moving object segmentation method. It takes range images and residual images as inputs and outputs the MOS results. (c) Our semantics-guided moving object segmentation method. It includes a single-scan-based semantic segmentation module and a multiple-scan-based MOS module. The latter takes range images and semantic features of the former as inputs. Note that to save space, only the middle size of 64 $\times$ 512 is displayed instead of the full size. This similarly applies to the rest figures of the paper.
• Figure 2: The flowchart of the proposed semantics-guided moving object segmentation method from LiDAR point cloud. The whole network consists of three modules and conducts single-scan-based semantic segmentation and multiple-scan-based moving object segmentation in a cascaded way. The semantic segmentation module learns the semantic features of each LiDAR point. The adjacent scan association module converts the features of different scans to the same coordinate system. The moving object segmentation module combines the transformed semantic features and LiDAR range images to differentiate the moving objects.
• Figure 3: The LiDAR range images include $range$, $x$, $y$, $z$ and $intensity$ components.
• Figure 4: The adjacent scan association module. The arrows denote the correspondences between adjacent LiDAR range images. From top to bottom are the current range image, previous range image, and the transformed range image.
• Figure 5: The range images in the current coordinate system. They are generated with different hardware (CPUs or GPUs) and LiDAR point clouds (current scan, previous scan, or previous range image). (a) and (b) are range images with current scan and based on CPUs and GPUs, respectively. (c) and (d) are range images with the previous scan and based on CPUs and GPUs, respectively. (e) and (f) are results with previous range images and based on CPUs and GPUs, respectively.