Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Human-Robot Teaming through Augmented Reality and Gaze-Based Attention Control

Yousra Shleibik, Elijah Alabi, Christopher Reardon

TL;DR

The paper addresses the challenge of efficient human-robot teaming in dynamic environments by enabling gaze-driven attention control through augmented reality. It proposes a system in which an AR headset displays dynamic visual markers projected by a ground robot, guiding user attention and adapting the robot’s level of intervention under a Variable Autonomy framework. Key contributions include formalizing attention attraction via gaze cues, integrating AR with gaze tracking for real-time attention management, and outlining an experimental plan with Raycasting-based gaze verification and initial two-user results. This approach has potential to improve situational awareness and collaboration efficiency in real-world HRT scenarios by providing intuitive, gaze-responsive guidance that decouples attention from direct mutual gaze or proximity cues.

Abstract

Robots are now increasingly integrated into various real world applications and domains. In these new domains, robots are mostly employed to improve, in some ways, the work done by humans. So, the need for effective Human-Robot Teaming (HRT) capabilities grows. These capabilities usually involve the dynamic collaboration between humans and robots at different levels of involvement, leveraging the strengths of both to efficiently navigate complex situations. Crucial to this collaboration is the ability of robotic systems to adjust their level of autonomy to match the needs of the task and the human team members. This paper introduces a system designed to control attention using HRT through the use of ground robots and augmented reality (AR) technology. Traditional methods of controlling attention, such as pointing, touch, and voice commands, sometimes fall short in precision and subtlety. Our system overcomes these limitations by employing AR headsets to display virtual visual markers. These markers act as dynamic cues to attract and shift human attention seamlessly, irrespective of the robot's physical location.

Towards Human-Robot Teaming through Augmented Reality and Gaze-Based Attention Control

TL;DR

The paper addresses the challenge of efficient human-robot teaming in dynamic environments by enabling gaze-driven attention control through augmented reality. It proposes a system in which an AR headset displays dynamic visual markers projected by a ground robot, guiding user attention and adapting the robot’s level of intervention under a Variable Autonomy framework. Key contributions include formalizing attention attraction via gaze cues, integrating AR with gaze tracking for real-time attention management, and outlining an experimental plan with Raycasting-based gaze verification and initial two-user results. This approach has potential to improve situational awareness and collaboration efficiency in real-world HRT scenarios by providing intuitive, gaze-responsive guidance that decouples attention from direct mutual gaze or proximity cues.

Abstract

Robots are now increasingly integrated into various real world applications and domains. In these new domains, robots are mostly employed to improve, in some ways, the work done by humans. So, the need for effective Human-Robot Teaming (HRT) capabilities grows. These capabilities usually involve the dynamic collaboration between humans and robots at different levels of involvement, leveraging the strengths of both to efficiently navigate complex situations. Crucial to this collaboration is the ability of robotic systems to adjust their level of autonomy to match the needs of the task and the human team members. This paper introduces a system designed to control attention using HRT through the use of ground robots and augmented reality (AR) technology. Traditional methods of controlling attention, such as pointing, touch, and voice commands, sometimes fall short in precision and subtlety. Our system overcomes these limitations by employing AR headsets to display virtual visual markers. These markers act as dynamic cues to attract and shift human attention seamlessly, irrespective of the robot's physical location.
Paper Structure (8 sections, 1 equation, 2 figures)

This paper contains 8 sections, 1 equation, 2 figures.

Table of Contents

  1. INTRODUCTION
  2. Attention Attraction and Shared Autonomy
  3. Approach
  4. Experimentation and Data Collection
  5. System Overview
  6. Experiment Design
  7. Initial results
  8. Conclusion and Future work:

Figures (2)

  • Figure 1: The system's overall structure involves an interaction between an AR device and an autonomous unmanned ground vehicle (UGV). The UGV continuously monitors the environment and sends marker information to the AR system. The AR system waits until it detects that the user has looked at the displayed marker (Marker 1) through gaze perception from the AR device. Once this is detected, the system processes the gaze data and moves the marker to another area of interest, as shown with Marker 2. This ensures that users are effectively guided to points of interest within their environment.
  • Figure 2: Attention attraction: (a) The robot sends an initial marker displayed by the AR system. (b) Once the user looks at the first marker, the system detects this and displays a second marker. (c) The system continues to show the second marker until it detects that the user is looking at it. (d) Then, it displays a third marker. (e) This process repeats, guiding the user through successive markers. (f) Finally, the user's attention is directed to the object of interest.