Asynchronous Training of Mixed-Role Human Actors in a Partially-Observable Environment

TL;DR

The paper addresses the challenge of teaching humans to coordinate in partially observable, mixed-role teams by introducing cooperative asynchronous training with autonomous surrogate teammates in HYBS. It presents a novel experimental design that clusters teammate behaviors from demonstrations to reduce the number of training conditions, and compares human-human training with apprentice and heuristic autonomous partners, plus a no-training baseline. While results show mixed outcomes—humans outperforming agents in training, and evaluation results showing no clear transfer—the study yields concrete design guidelines for building and evaluating cooperative asynchronous training systems and robot surrogates. The work has practical implications for scalable, time-efficient training of human teams in complex, information-fragmented environments and informs future development of autonomous training partners.

Abstract

In cooperative training, humans within a team coordinate on complex tasks, building mental models of their teammates and learning to adapt to teammates' actions in real-time. To reduce the often prohibitive scheduling constraints associated with cooperative training, this article introduces a paradigm for cooperative asynchronous training of human teams in which trainees practice coordination with autonomous teammates rather than humans. We introduce a novel experimental design for evaluating autonomous teammates for use as training partners in cooperative training. We apply the design to a human-subjects experiment where humans are trained with either another human or an autonomous teammate and are evaluated with a new human subject in a new, partially observable, cooperative game developed for this study. Importantly, we employ a method to cluster teammate trajectories from demonstrations performed in the experiment to form a smaller number of training conditions. This results in a simpler experiment design that enabled us to conduct a complex cooperative training human-subjects study in a reasonable amount of time. Through a demonstration of the proposed experimental design, we provide takeaways and design recommendations for future research in the development of cooperative asynchronous training systems utilizing robot surrogates for human teammates.

Figures (4)

• Figure 1: Visualization of gameplay in the HYBS partially-observable multi-agent teaming game. The novel waiter role assigns a priority to each customer and recommends a dish for them to be served, which the chef role uses to plan and execute dish servings in a round. The waiter and chef have different sets of information and must coordinate in pursuit of tip rewards by serving dishes to customers matching those customers' preferences.
• Figure 2: Behavior clustering obtained from $K$-Medoids applied to sequential encodings of established chef demonstrations.
• Figure 3: Overview of proposed asynchronous cooperative training experiment. Established chefs train via self-play, producing data used to learn clusters of commonly-expressed chef behaviors and train agent chefs. Waiters are randomly assigned a training partner---a human or agent (apprentice or heuristic) chef---and cluster. A control condition receives no training treatment. All chefs are evaluated with an established human chef from the cluster they trained with and one from an unseen cluster.
• Figure 4: Training and evaluation metrics obtained for the demonstrated human-subjects experiment. Values were normalized to range [0,1], as described in Section \ref{['sec: metrics']} along with metric definitions. Significant pairwise comparisons are indicated with * if $p\in[0.01,0.05)$, ** if $p\in[0.001,0.01)$, and *** if $p<0.001$. Because evaluation tip in the NULL condition was not significantly different than that of other conditions, we find a negative result for research question Q3 (Benefit); thus, this study is not equipped to answer research questions Q1 (Effectiveness) or Q2 (Personalization).