Robust Onboard Localization in Changing Environments Exploiting Text Spotting

TL;DR

This paper addresses the problem of localizing in an indoor environment that changes and where prominent structures have no correspondence in the map built at a different point in time with a Monte Carlo localization framework using a particle filter that operates on 2D LiDAR scans and camera data.

Abstract

Robust localization in a given map is a crucial component of most autonomous robots. In this paper, we address the problem of localizing in an indoor environment that changes and where prominent structures have no correspondence in the map built at a different point in time. To overcome the discrepancy between the map and the observed environment caused by such changes, we exploit human-readable localization cues to assist localization. These cues are readily available in most facilities and can be detected using RGB camera images by utilizing text spotting. We integrate these cues into a Monte Carlo localization framework using a particle filter that operates on 2D LiDAR scans and camera data. By this, we provide a robust localization solution for environments with structural changes and dynamics by humans walking. We evaluate our localization framework on multiple challenging indoor scenarios in an office environment. The experiments suggest that our approach is robust to structural changes and can run on an onboard computer. We release an open source implementation of our approach (upon paper acceptance), which uses off-the-shelf text spotting, written in C++ with a ROS wrapper.

Figures (8)

• Figure 1: Top Left: The corridor in which the experiment took place in. Top right: The Kuka YouBot platform that was used for data collection, equipped with 2D LiDAR scanners and cameras that cover the complete $360^{\circ}$ field-of-view we utilize for text spotting. Bottom: The results of of localization in a corridor with closed doors (indicated by red lines), which are not reflected in the map, with and without textual cues.
• Figure 2: The text likelihood maps, based on the collected data, indicate the locations in which detection of each room number is likely. The likelihood maps are used for particle injection when a detection of a known text cues occurs.
• Figure 3: Particle injection with text spotting. (a) Before detection, we have a situation with multi-modal distribution of particles (shown in red) as the corridor with closed doors is a symmetric situation that cannot be resolved just using the LiDAR scans. (b) With the first text detection (indicated by the green cross), we can inject new particles inside the bounding box extracted from the text map. We replace low weighted particles by new particles (shown in blue) that are uniformly distributed inside the corresponding bounding box of the text detection (shown by a dashed green line).
• Figure 4: The data collection platform, an omnidirectional Kuka YouBot, with 2D LiDAR scanners (marked by a red outline) and with 4 cameras (marked by a blue outline) providing $360^{\circ}$ coverage. The up-ward facing camera (marked by a green outline) is only used for generating the ground truth via AprilTag detections.
• Figure 5: Visualization of the different sequences used for evaluating our approach. Sequences S1-S10 correspond to the scenario where all doors are closed. Sequences D1-D4 were recorded with all the doors open, and with moderate amount of humans moving around. The color of the trajectory correspond to the time, where purple is the beginning and red corresponds to the end of the sequence.