Training Human-Robot Teams by Improving Transparency Through a Virtual Spectator Interface

TL;DR

This work introduces the Virtual Spectator Interface (VSI) as a training review tool to increase transparency of robotic teammates in human-robot teams and evaluates its effectiveness in a simulated search mission. A 1 × 3 between-subjects study compares VSI-based TR, screen-record TR, and verbal-only TR, measuring task performance and situation awareness (SA) across take-over, object-flagging, and secondary tasks. Results show TR improves performance overall, but VSI does not significantly outperform other TR formats on aggregate; however, VSI confers a notable SA improvement for participants with initially lower SA, suggesting targeted benefits in underperforming subgroups. The findings imply VSI’s potential for enhancing SA in low-SA individuals and highlight the need for longer, more immersive studies that optimize information content and incorporate physiological metrics to maximize TR effectiveness in human-robot teams.

Abstract

After-action reviews (AARs) are professional discussions that help operators and teams enhance their task performance by analyzing completed missions with peers and professionals. Previous studies that compared different formats of AARs have mainly focused on human teams. However, the inclusion of robotic teammates brings along new challenges in understanding teammate intent and communication. Traditional AAR between human teammates may not be satisfactory for human-robot teams. To address this limitation, we propose a new training review (TR) tool, called the Virtual Spectator Interface (VSI), to enhance human-robot team performance and situational awareness (SA) in a simulated search mission. The proposed VSI primarily utilizes visual feedback to review subjects' behavior. To examine the effectiveness of VSI, we took elements from AAR to conduct our own TR, designed a 1 x 3 between-subjects experiment with experimental conditions: TR with (1) VSI, (2) screen recording, and (3) non-technology (only verbal descriptions). The results of our experiments demonstrated that the VSI did not result in significantly better team performance than other conditions. However, the TR with VSI led to more improvement in the subjects SA over the other conditions.

Figures (4)

• Figure 1: Examples of the Virtual Spectator Interface (VSI): (a) UI for input overlay and replay timeline and (b) UI for highlighting obstacle, autonomy, and road. The color used for highlighting represents, (c) the hazardous levels (HL) on roads, (d) the status of the flagging object, and (e) the status of UGV's autonomy. The supplementary video of the VSI can be found at https://sites.google.com/umich.edu/mavric/projects/arc_sasi.
• Figure 2: Details of a driving simulator controlling UGVs and the main experiment UI streaming the cameras of two UGVs and SR task on the large monitor.
• Figure 3: Map illustrating the distribution of takeover tasks, SAGAT pauses, and object flagging tasks in Trial 2 map. More details of the maps used in Trial 1 and 2 can be found in this supplementary website: https://sites.google.com/umich.edu/mavric/projects/arc_sasi.
• Figure 4: The data distribution of $\Delta SA$ from a subgroup with scores lower than the mean of $SA$ in $T_{1}$ for all subjects. The white triangle indicates the mean.