Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hector UI: A Flexible Human-Robot User Interface for (Semi-)Autonomous Rescue and Inspection Robots

Stefan Fabian, Oskar von Stryk

TL;DR

This work addresses the need for user-centered, flexible UIs in rescue and inspection robotics, where end-user operators—often non-experts—must interact with semi-autonomous systems under high-stress conditions. The authors implement an open-source UI overlay on RViz using the hector_rviz_overlay framework, combining a Dashboard and a 3D View to present status, sensors, and control in a unified, low-clutter window. Key contributions include a modular, configurable architecture with three control levels (Supervisory, Direct, High-Level), robust state management via singletons, and practical mechanisms for adapting to varied platforms and sensors, demonstrated across ARGOS, WRS, RoboCup Rescue, ENRICH, and a field deployment. The approach offers actionable guidance for developing flexible, operator-friendly interfaces in real-world robotic rescue and inspection tasks, potentially reducing operator workload and improving mission success. The open-source release enables rapid adoption and extension to new robots and scenarios, enhancing the impact of robotics in emergency response and industrial inspection contexts.

Abstract

The remote human operator's user interface (UI) is an important link to make the robot an efficient extension of the operator's perception and action. In rescue applications, several studies have investigated the design of operator interfaces based on observations during major robotics competitions or field deployments. Based on this research, guidelines for good interface design were empirically identified. The investigations on the UIs of teams participating in competitions are often based on external observations during UI application, which may miss some relevant requirements for UI flexibility. In this work, we present an open-source and flexibly configurable user interface based on established guidelines and its exemplary use for wheeled, tracked, and walking robots. We explain the design decisions and cover the insights we have gained during its highly successful applications in multiple robotics competitions and evaluations. The presented UI can also be adapted for other robots with little effort and is available as open source.

Hector UI: A Flexible Human-Robot User Interface for (Semi-)Autonomous Rescue and Inspection Robots

TL;DR

This work addresses the need for user-centered, flexible UIs in rescue and inspection robotics, where end-user operators—often non-experts—must interact with semi-autonomous systems under high-stress conditions. The authors implement an open-source UI overlay on RViz using the hector_rviz_overlay framework, combining a Dashboard and a 3D View to present status, sensors, and control in a unified, low-clutter window. Key contributions include a modular, configurable architecture with three control levels (Supervisory, Direct, High-Level), robust state management via singletons, and practical mechanisms for adapting to varied platforms and sensors, demonstrated across ARGOS, WRS, RoboCup Rescue, ENRICH, and a field deployment. The approach offers actionable guidance for developing flexible, operator-friendly interfaces in real-world robotic rescue and inspection tasks, potentially reducing operator workload and improving mission success. The open-source release enables rapid adoption and extension to new robots and scenarios, enhancing the impact of robotics in emergency response and industrial inspection contexts.

Abstract

The remote human operator's user interface (UI) is an important link to make the robot an efficient extension of the operator's perception and action. In rescue applications, several studies have investigated the design of operator interfaces based on observations during major robotics competitions or field deployments. Based on this research, guidelines for good interface design were empirically identified. The investigations on the UIs of teams participating in competitions are often based on external observations during UI application, which may miss some relevant requirements for UI flexibility. In this work, we present an open-source and flexibly configurable user interface based on established guidelines and its exemplary use for wheeled, tracked, and walking robots. We explain the design decisions and cover the insights we have gained during its highly successful applications in multiple robotics competitions and evaluations. The presented UI can also be adapted for other robots with little effort and is available as open source.
Paper Structure (19 sections, 10 figures)

This paper contains 19 sections, 10 figures.

Table of Contents

  1. INTRODUCTION
  2. DESIGN
  3. Overview
  4. Dashboard
  5. 3D View
  6. Dashboard
  7. 3D View
  8. CONTROL
  9. Supervisory Control
  10. Direct Control
  11. High-Level Control
  12. IMPLEMENTATION
  13. Framework
  14. Configuration
  15. APPLICATIONS
  16. ...and 4 more sections

Figures (10)

  • Figure 1: The open-sourced interface presented in this paper in use at the international RoboCup Rescue 2023 competition.
  • Figure 2: The operator interface for our robot Asterix while the UI E-Stop is active.
  • Figure 3: The eC Scout robot UI while exploring and mapping a part of a nuclear plant, and locating radioactive probes in the context of the ENRICH 2023 robotics hackathon.
  • Figure 4: The gamepad configurations for driving the robot (left) and using the gripper to inspect or manipulate objects (right).
  • Figure 5: The action status during execution (a) and after term-ination with status canceled (b), successful (c) or failed (d).
  • ...and 5 more figures