Influence of Operator Expertise on Robot Supervision and Intervention

TL;DR

This study investigates how operator expertise affects supervision and intervention in semi-autonomous remote robots navigating unknown tunnels. Using a NavStack-based simulator with four tunnel scenarios and a single intervention per mission, the authors analyze timing and waypoint strategies across Novice, Intermediate, and Experienced operators (N=27). They find that novices intervene earlier and rely on clear cues, while experts wait longer and place more effective waypoints, with robot autonomy outperforming human intervention in at least one scenario. The findings inform the design of adaptive shared autonomy that estimates operator expertise in real time and tailors autonomy and prompts to novices and experts alike, aiming to improve human-robot collaboration in diverse user populations.

Abstract

With increasing levels of robot autonomy, robots are increasingly being supervised by users with varying levels of robotics expertise. As the diversity of the user population increases, it is important to understand how users with different expertise levels approach the supervision task and how this impacts performance of the human-robot team. This exploratory study investigates how operators with varying expertise levels perceive information and make intervention decisions when supervising a remote robot. We conducted a user study (N=27) where participants supervised a robot autonomously exploring four unknown tunnel environments in a simulator, and provided waypoints to intervene when they believed the robot had encountered difficulties. By analyzing the interaction data and questionnaire responses, we identify differing patterns in intervention timing and decision-making strategies across novice, intermediate, and expert users.

Figures (5)

• Figure 1: Four scenarios with annotated challenging tunnel maps. The robot's trajectory (red line) is extracted from recorded data. Red triangle marks trajectory start and red star marks the end. Dark blue annotations indicate points of robot delays. Predefined failure points, requiring human intervention, are marked at location c in scenarios 1-3 and b in scenario 4.
• Figure 2: Response Times by Scenario and Group
• Figure 3: User intervention relating to robot trajectory. Waypoints by Novices, Intermediates, and Experts are marked as orange, green, and purple circles, respectively, with arrows indicating mean orientation. Corresponding robot positions are shown as squares in the same colors. Variance ellipses depict the spread of intervention points. The robot's trajectory is plotted in red (arrow for start and X for end).
• Figure 4: Area covered vs. time for the expertise group and the autonomous robot baseline. The plot compares area coverage over time for each novice, intermediate, and experienced users, along with the autonomous robot baseline.
• Figure 5: Mean differences in area covered by groups. The x-axis shows the area covered. The y-axis lists Robot, Novice, Intermediate, and Experienced groups. Horizontal lines indicate confidence intervals, with central dots being mean values.