Semantic2D: Enabling Semantic Scene Understanding with 2D Lidar Alone

TL;DR

The paper tackles the challenge of semantic scene understanding using only 2D lidar by introducing a complete workflow tailored for indoor mobile robotics. It presents Semantic2D, the first public 2D lidar semantic dataset; SALSA, a semi-automatic labeling framework; and S$^3$-Net, a fast, stochastic segmentation network built on a Variational Autoencoder. The authors demonstrate semantic occupancy grid mapping and two semantic-aware navigation policies, showing improved performance and cross-platform generalization, while also providing open-source code and data. This camera-free approach enables robust semantic perception for low-cost, privacy-preserving robotic systems and can enhance a range of lidar-based perception and control tasks.

Abstract

This article presents a complete semantic scene understanding workflow using only a single 2D lidar. This fills the gap in 2D lidar semantic segmentation, thereby enabling the rethinking and enhancement of existing 2D lidar-based algorithms for application in various mobile robot tasks. It introduces the first publicly available 2D lidar semantic segmentation dataset and the first fine-grained semantic segmentation algorithm specifically designed for 2D lidar sensors on autonomous mobile robots. To annotate this dataset, we propose a novel semi-automatic semantic labeling framework that requires minimal human effort and provides point-level semantic annotations. The data was collected by three different types of 2D lidar sensors across twelve indoor environments, featuring a range of common indoor objects. Furthermore, the proposed semantic segmentation algorithm fully exploits raw lidar information -- position, range, intensity, and incident angle -- to deliver stochastic, point-wise semantic segmentation. We present a series of semantic occupancy grid mapping experiments and demonstrate two semantically-aware navigation control policies based on 2D lidar. These results demonstrate that the proposed semantic 2D lidar dataset, semi-automatic labeling framework, and segmentation algorithm are effective and can enhance different components of the robotic navigation pipeline. Multimedia resources are available at: https://youtu.be/P1Hsvj6WUSY.

Figures (20)

• Figure 1: Workflow for 2D lidar semantic segmentation, enabling enhanced scene understanding for mobile robotics.
• Figure 2: Floor plans of the dataset collection environments, depicting nine indoor settings across four buildings at Temple University and three additional environments from three buildings at the University of Hong Kong.
• Figure 3: The percentage of each class in the Semantic2D dataset.
• Figure 4: SALSA: a semi-automatic semantic labeling framework for the Semantic2D dataset that only requires manual labeling of a pre-mapped environment.
• Figure 5: Semantic label visualization for the Semantic2D dataset, with color-coded class assignments