Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HEROES: Unreal Engine-based Human and Emergency Robot Operation Education System

Anav Chaudhary, Kshitij Tiwari, Aniket Bera

TL;DR

The proposed HEROES simulator is a versatile Unreal Engine-based simulator capable of generating synthetic datasets for machine learning pipelines that are used for training robot navigation and has been integrated with ROS and has been used to train an RL model for a real robot as a proof of concept.

Abstract

Training and preparing first responders and humanitarian robots for Mass Casualty Incidents (MCIs) often poses a challenge owing to the lack of realistic and easily accessible test facilities. While such facilities can offer realistic scenarios post an MCI that can serve training and educational purposes for first responders and humanitarian robots, they are often hard to access owing to logistical constraints. To overcome this challenge, we present HEROES- a versatile Unreal Engine simulator for designing novel training simulations for humans and emergency robots for such urban search and rescue operations. The proposed HEROES simulator is capable of generating synthetic datasets for machine learning pipelines that are used for training robot navigation. This work addresses the necessity for a comprehensive training platform in the robotics community, ensuring pragmatic and efficient preparation for real-world emergency scenarios. The strengths of our simulator lie in its adaptability, scalability, and ability to facilitate collaboration between robot developers and first responders, fostering synergy in developing effective strategies for search and rescue operations in MCIs. We conducted a preliminary user study with an 81% positive response supporting the ability of HEROES to generate sufficiently varied environments, and a 78% positive response affirming the usefulness of the simulation environment of HEROES.

HEROES: Unreal Engine-based Human and Emergency Robot Operation Education System

TL;DR

The proposed HEROES simulator is a versatile Unreal Engine-based simulator capable of generating synthetic datasets for machine learning pipelines that are used for training robot navigation and has been integrated with ROS and has been used to train an RL model for a real robot as a proof of concept.

Abstract

Training and preparing first responders and humanitarian robots for Mass Casualty Incidents (MCIs) often poses a challenge owing to the lack of realistic and easily accessible test facilities. While such facilities can offer realistic scenarios post an MCI that can serve training and educational purposes for first responders and humanitarian robots, they are often hard to access owing to logistical constraints. To overcome this challenge, we present HEROES- a versatile Unreal Engine simulator for designing novel training simulations for humans and emergency robots for such urban search and rescue operations. The proposed HEROES simulator is capable of generating synthetic datasets for machine learning pipelines that are used for training robot navigation. This work addresses the necessity for a comprehensive training platform in the robotics community, ensuring pragmatic and efficient preparation for real-world emergency scenarios. The strengths of our simulator lie in its adaptability, scalability, and ability to facilitate collaboration between robot developers and first responders, fostering synergy in developing effective strategies for search and rescue operations in MCIs. We conducted a preliminary user study with an 81% positive response supporting the ability of HEROES to generate sufficiently varied environments, and a 78% positive response affirming the usefulness of the simulation environment of HEROES.
Paper Structure (19 sections, 2 equations, 4 figures, 3 tables)

This paper contains 19 sections, 2 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Training professionals for MCIs through computer-aided simulations
  4. Using computer simulations to train robot navigation
  5. Use of simulation in training robot navigation for MCI environments
  6. HEROES: Post-MCI Environment Simulator
  7. Individual destructible unit
  8. Simulation environment
  9. Destruction events
  10. ROS integration
  11. Experiments
  12. Experiment tasks
  13. Usability survey
  14. Real-World Implementation and Testing
  15. Leveraging 3D Simulated Data for Training
  16. ...and 4 more sections

Figures (4)

  • Figure 1: Urban post-MCI destruction environments are not represented during robot navigation training and yet offer unique challenges. The figure showcases the highly irregular environment created by our proposed simulator HEROES with a wide variety of lighting scenarios due to structural collapse post MCIs. In addition, the figure showcases the type of perceptual feedback that can be extracted, ordered from left to right, such as Color Image, Depth, and Semantic Segmentation. Finally, the figure also showcases how the simulator is being used to train robots to tackle similar environments in the real world.
  • Figure 2: Multiple different types of Rooms can be used to construct the simulation environment. In a clockwise order, a simple Room with 1 doorway, an L-shaped Room with 1 doorway and 1 window, and translucent views of a 1 doorway room with 2 pillars for support, and a 1 doorway room with a beam and two wall supports are shown.
  • Figure 3: Different forms of destruction events can be generated via the HEROES simulator. Left: Earthquake-sourced strain, Middle: damage due to explosion, Right: constrained building collapse.
  • Figure 4: The ROS Integration allows us to transfer data between the simulator and ROS, which is one of the most popular software development platforms for robotics. The images above show the different forms of data that can be exported from the simulator as images of a quadrupedal robot's point of view. They are arranged as follows; Left: Color Image, Middle: Depth Image, Right: Semantic Segmentation.