Automated Assessment and Adaptive Multimodal Formative Feedback Improves Psychomotor Skills Training Outcomes in Quadrotor Teleoperation

TL;DR

This work tackles scalable upskilling for humans working with autonomous systems by developing an end-to-end system that automatically assesses quadrotor-landing performance via temporal-logic task specifications and generates formative, multimodal feedback using AI. It compares a baseline of summary metrics, AI-generated text feedback, and AI-generated text with an annotated trajectory image, finding that multimodal feedback leads to higher safe-landing rates and larger performance gains over time. The study contributes a flexible framework combining formal task specifications with pedagogy-grounded feedback and provides evidence that feedback content and presentation significantly affect learning in complex psychomotor tasks. The results support the practical potential of adaptive, theory-informed training to scale up industrial upskilling and improve operator performance in human-robot teams.

Abstract

The workforce will need to continually upskill in order to meet the evolving demands of industry, especially working with robotic and autonomous systems. Current training methods are not scalable and do not adapt to the skills that learners already possess. In this work, we develop a system that automatically assesses learner skill in a quadrotor teleoperation task using temporal logic task specifications. This assessment is used to generate multimodal feedback based on the principles of effective formative feedback. Participants perceived the feedback positively. Those receiving formative feedback viewed the feedback as more actionable compared to receiving summary statistics. Participants in the multimodal feedback condition were more likely to achieve a safe landing and increased their safe landings more over the experiment compared to other feedback conditions. Finally, we identify themes to improve adaptive feedback and discuss and how training for complex psychomotor tasks can be integrated with learning theories.

Figures (3)

• Figure 1: Screenshot of the starting configuration of the quadrotor landing task.
• Figure 2: Examples of the three feedback conditions used in the experiment.
• Figure 3: Distributions of responses for feedback dimensions that were significantly different between groups. We show the responses collapsed to a three-point likert scale.