Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integration of Augmented Reality and Mobile Robot Indoor SLAM for Enhanced Spatial Awareness

Michael D. Friske

TL;DR

The paper addresses the challenge of enhancing situational awareness in hazardous indoor environments by fusing mobile robot SLAM with Augmented Reality (AR) to provide real-time spatial perception to non-co-located users. It proposes a ROS2-based, three-node architecture (robot, base station, AR device), implements a Unity-based AR application on Android, and uses AprilTags for initial pose alignment and ARCore for pose estimation. The study compares ORB-SLAM3 and RTAB-Map for visualization, develops a custom tracked robot, and evaluates data streaming, rendering performance, and visualization strategies across four experiments. The findings indicate that RTAB-Map yields more intuitive, dense, and color-rich AR visualizations, while minimizing CPU-GPU data transfers and leveraging robust streaming pipelines to improve safety and efficiency in rescue operations. Overall, the work demonstrates a viable pathway to see-through-wall situational awareness and informs future enhancements like collaborative SLAM, onboard processing, and multimodal sensing for indoor robotics and AR applications.

Abstract

This research explores the integration of indoor Simultaneous Localization and Mapping (SLAM) with Augmented Reality (AR) to enhance situational awareness, improving safety in hazardous or emergency situations. The main contribution of this work is to enable mobile robots to provide real-time spatial perception to users who are not co-located with the robot. This is a comprehensive approach, including selecting suitable sensors for indoor SLAM, designing and building a platform, developing methods to display maps on AR devices, implementing this into software on an AR device, and improving the robustness of communication and localization between the robot and AR device in real-world testing. By taking this approach and analyzing each component of the integrated system, this paper highlights numerous areas for future research that can further advance the integration of SLAM and AR technologies. These advancements aim to significantly improve safety and efficiency during rescue operations.

Integration of Augmented Reality and Mobile Robot Indoor SLAM for Enhanced Spatial Awareness

TL;DR

The paper addresses the challenge of enhancing situational awareness in hazardous indoor environments by fusing mobile robot SLAM with Augmented Reality (AR) to provide real-time spatial perception to non-co-located users. It proposes a ROS2-based, three-node architecture (robot, base station, AR device), implements a Unity-based AR application on Android, and uses AprilTags for initial pose alignment and ARCore for pose estimation. The study compares ORB-SLAM3 and RTAB-Map for visualization, develops a custom tracked robot, and evaluates data streaming, rendering performance, and visualization strategies across four experiments. The findings indicate that RTAB-Map yields more intuitive, dense, and color-rich AR visualizations, while minimizing CPU-GPU data transfers and leveraging robust streaming pipelines to improve safety and efficiency in rescue operations. Overall, the work demonstrates a viable pathway to see-through-wall situational awareness and informs future enhancements like collaborative SLAM, onboard processing, and multimodal sensing for indoor robotics and AR applications.

Abstract

This research explores the integration of indoor Simultaneous Localization and Mapping (SLAM) with Augmented Reality (AR) to enhance situational awareness, improving safety in hazardous or emergency situations. The main contribution of this work is to enable mobile robots to provide real-time spatial perception to users who are not co-located with the robot. This is a comprehensive approach, including selecting suitable sensors for indoor SLAM, designing and building a platform, developing methods to display maps on AR devices, implementing this into software on an AR device, and improving the robustness of communication and localization between the robot and AR device in real-world testing. By taking this approach and analyzing each component of the integrated system, this paper highlights numerous areas for future research that can further advance the integration of SLAM and AR technologies. These advancements aim to significantly improve safety and efficiency during rescue operations.
Paper Structure (18 sections, 12 figures)

This paper contains 18 sections, 12 figures.

Table of Contents

  1. Introduction
  2. Objectives
  3. Sensors
  4. Simultaneous Localization and Mapping
  5. Mobile Robot
  6. Software
  7. General Architecture
  8. Mobile Robot Architecture
  9. PC Base Station Architecture
  10. AR Application
  11. Experimentation
  12. Webcam and Streaming Performance
  13. AR Application Performance
  14. AprilTag Position Averaging
  15. Displaying AR Maps
  16. ...and 3 more sections

Figures (12)

  • Figure 1: Sensors on the Robot Tank
  • Figure 2: Robot Block Diagram
  • Figure 3: Final Design
  • Figure 4: Overall Architecture
  • Figure 5: Mobile Robot Architecture
  • ...and 7 more figures