Designs for Enabling Collaboration in Human-Machine Teaming via Interactive and Explainable Systems

TL;DR

This study develops HMT approaches that enable iterative, mixed-initiative team development allowing end-users to interactively reprogram interpretable AI teammates and presents three important future research directions: improving the ability to generate collaborative agents with white-box models, better learning methods to facilitate collaboration rather than individualized coordination, and Mixed-initiative interfaces that enable users, who may vary in ability, to improve collaboration.

Abstract

Collaborative robots and machine learning-based virtual agents are increasingly entering the human workspace with the aim of increasing productivity and enhancing safety. Despite this, we show in a ubiquitous experimental domain, Overcooked-AI, that state-of-the-art techniques for human-machine teaming (HMT), which rely on imitation or reinforcement learning, are brittle and result in a machine agent that aims to decouple the machine and human's actions to act independently rather than in a synergistic fashion. To remedy this deficiency, we develop HMT approaches that enable iterative, mixed-initiative team development allowing end-users to interactively reprogram interpretable AI teammates. Our 50-subject study provides several findings that we summarize into guidelines. While all approaches underperform a simple collaborative heuristic (a critical, negative result for learning-based methods), we find that white-box approaches supported by interactive modification can lead to significant team development, outperforming white-box approaches alone, and that black-box approaches are easier to train and result in better HMT performance highlighting a tradeoff between explainability and interactivity versus ease-of-training. Together, these findings present three important future research directions: 1) Improving the ability to generate collaborative agents with white-box models, 2) Better learning methods to facilitate collaboration rather than individualized coordination, and 3) Mixed-initiative interfaces that enable users, who may vary in ability, to improve collaboration.

Figures (9)

• Figure 1: Case Study in Human-Machine Teaming with Different Teaming Strategies. It is clear that the models are not robust to multiple strategies of play and can result in agents performing nonsensical behavior (e.g., stuck in place).
• Figure 2: Here, we provide an overview of the steps to produce a collaborative AI teammate with an interpretable policy and the proposed policy modification scheme evaluated in our user study.
• Figure 3: Users have several capabilities in creating an effective teammate, including modifying the tree structure by adding or removing decision nodes, changing state features the tree is conditioned on, and modifying actions and/or their respective probabilities at leaf nodes.
• Figure 4: User gameplay scores across teaming iterations with per-iteration means connected by the red dotted line and the per-iteration standard deviation shaded in red.
• Figure 5: Maximum Reward and Subjective Ratings Across IV1 Factors.