uPLAM: Robust Panoptic Localization and Mapping Leveraging Perception Uncertainties

TL;DR

This work presents uncertainty-aware Panoptic Localization and Mapping (uPLAM), which employs pixel-wise uncertainty estimates for panoptic CNNs as a bridge to fuse modern perception with classical probabilistic localization and mapping approaches.

Abstract

The availability of a robust map-based localization system is essential for the operation of many autonomously navigating vehicles. Since uncertainty is an inevitable part of perception, it is beneficial for the robustness of the robot to consider it in typical downstream tasks of navigation stacks. In particular localization and mapping methods, which in modern systems often employ convolutional neural networks (CNNs) for perception tasks, require proper uncertainty estimates. In this work, we present uncertainty-aware Panoptic Localization and Mapping (uPLAM), which employs pixel-wise uncertainty estimates for panoptic CNNs as a bridge to fuse modern perception with classical probabilistic localization and mapping approaches. Beyond the perception, we introduce an uncertainty-based map aggregation technique to create accurate panoptic maps, containing surface semantics and landmark instances. Moreover, we provide cell-wise map uncertainties, and present a particle filter-based localization method that employs perception uncertainties. Extensive evaluations show that our proposed incorporation of uncertainties leads to more accurate maps with reliable uncertainty estimates and improved localization accuracy. Additionally, we present the Freiburg Panoptic Driving dataset for evaluating panoptic mapping and localization methods. We make our code and dataset available at: \url{http://uplam.cs.uni-freiburg.de}

Figures (6)

• Figure 1: Our panoptic map (upper) with associated cell-wise uncertainties (lower). The map contains surface semantics for drivable regions (purple) and road markings (yellow), together with landmark instances for traffic signs (blue) and traffic lights (green). We also show the predicted trajectory (red) and the particle cloud (cyan) of our uncertainty-aware panoptic localization method.
• Figure 2: Overview of our proposed uncertainty-aware panoptic mapping and localization approach.
• Figure 3: Example labeled images of the Freiburg Panoptic Driving dataset.
• Figure 4: Localization errors for a single trajectory.
• Figure 5: Qualitative results for our uncertainty-aware panoptic mapping method on the Freiburg data.