Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding why SLAM algorithms fail in modern indoor environments

Nwankwo Linus, Elmar Rueckert

TL;DR

SLAM in modern indoor environments suffers from dynamic objects, transparent and reflective surfaces, and illumination changes, causing failures in localization and mapping. The paper proposes a real-world evaluation strategy and curated indoor datasets to benchmark five state-of-the-art SLAM systems (three lidar/graph-based: Hector-SLAM, Gmapping, Karto-SLAM; two visual-based: RTAB-Map, ORB-SLAM2) across Absolute Trajectory Error, Relative Pose Error, Scale Drift, and map quality. Results show substantial robustness gaps in challenging settings, with loop closures significantly shaping performance; Hector-SLAM excels in long, odometry-insensitive scenarios, while others are hampered by dynamic objects and reflective surfaces. The findings motivate further robustness improvements and suggest augmenting SLAM with priors such as floor plans to enhance loop closure and map consistency in real-world deployments.

Abstract

Simultaneous localization and mapping (SLAM) algorithms are essential for the autonomous navigation of mobile robots. With the increasing demand for autonomous systems, it is crucial to evaluate and compare the performance of these algorithms in real-world environments. In this paper, we provide an evaluation strategy and real-world datasets to test and evaluate SLAM algorithms in complex and challenging indoor environments. Further, we analysed state-of-the-art (SOTA) SLAM algorithms based on various metrics such as absolute trajectory error, scale drift, and map accuracy and consistency. Our results demonstrate that SOTA SLAM algorithms often fail in challenging environments, with dynamic objects, transparent and reflecting surfaces. We also found that successful loop closures had a significant impact on the algorithm's performance. These findings highlight the need for further research to improve the robustness of the algorithms in real-world scenarios.

Understanding why SLAM algorithms fail in modern indoor environments

TL;DR

SLAM in modern indoor environments suffers from dynamic objects, transparent and reflective surfaces, and illumination changes, causing failures in localization and mapping. The paper proposes a real-world evaluation strategy and curated indoor datasets to benchmark five state-of-the-art SLAM systems (three lidar/graph-based: Hector-SLAM, Gmapping, Karto-SLAM; two visual-based: RTAB-Map, ORB-SLAM2) across Absolute Trajectory Error, Relative Pose Error, Scale Drift, and map quality. Results show substantial robustness gaps in challenging settings, with loop closures significantly shaping performance; Hector-SLAM excels in long, odometry-insensitive scenarios, while others are hampered by dynamic objects and reflective surfaces. The findings motivate further robustness improvements and suggest augmenting SLAM with priors such as floor plans to enhance loop closure and map consistency in real-world deployments.

Abstract

Simultaneous localization and mapping (SLAM) algorithms are essential for the autonomous navigation of mobile robots. With the increasing demand for autonomous systems, it is crucial to evaluate and compare the performance of these algorithms in real-world environments. In this paper, we provide an evaluation strategy and real-world datasets to test and evaluate SLAM algorithms in complex and challenging indoor environments. Further, we analysed state-of-the-art (SOTA) SLAM algorithms based on various metrics such as absolute trajectory error, scale drift, and map accuracy and consistency. Our results demonstrate that SOTA SLAM algorithms often fail in challenging environments, with dynamic objects, transparent and reflecting surfaces. We also found that successful loop closures had a significant impact on the algorithm's performance. These findings highlight the need for further research to improve the robustness of the algorithms in real-world scenarios.
Paper Structure (10 sections, 4 equations, 2 figures, 1 table)

This paper contains 10 sections, 4 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Hardware Platforms
  5. SLAM Algorithms
  6. Environment and Data Collection
  7. Metrics
  8. Results
  9. Lidar-based and Visual-based SLAM Techniques
  10. Conclusion

Figures (2)

  • Figure 1: Illustration of the environments. In the areas marked in red in (a) and (b), the algorithm could not track the robot's trajectory. The algorithm failed to create a consistent metric map as a result of the accumulation of odometry error and insufficient geometric features for the pose estimation due to the reflectivity of glass walls.
  • Figure 2: ATE result for all the environments. In each plot, the dashed vertical lines represent the mean ATE for each algorithm.