Model Cards for AI Teammates: Comparing Human-AI Team Familiarization Methods for High-Stakes Environments

TL;DR

This study addresses how to effectively familiarize humans with autonomous AI teammates in high-stakes ISR tasks by comparing documentation-based, in-situ training, and a control approach. Using a 2D ISR game with a heuristic AI, the authors measure performance, strategies, SA, workload, trust, and AI understanding across 60 participants. They find that knowledge of the AI's decision-making and relative strengths/weaknesses is most valuable, with documentation enabling the fastest strategy adoption but sometimes biasing toward risk aversion, while in-situ training supports exploratory behavior and practical understanding but yields a weaker internal model of the AI. The authors conclude that a hybrid familiarization approach—combining model-card-style documentation, structured in-situ training, and exploratory interaction—best supports human-AI teamwork in high-stakes environments, while accounting for individual differences in risk tolerance and preferred control levels.

Abstract

We compare three methods of familiarizing a human with an artificial intelligence (AI) teammate ("agent") prior to operation in a collaborative, fast-paced intelligence, surveillance, and reconnaissance (ISR) environment. In a between-subjects user study (n=60), participants either read documentation about the agent, trained alongside the agent prior to the mission, or were given no familiarization. Results showed that the most valuable information about the agent included details of its decision-making algorithms and its relative strengths and weaknesses compared to the human. This information allowed the familiarization groups to form sophisticated team strategies more quickly than the control group. Documentation-based familiarization led to the fastest adoption of these strategies, but also biased participants towards risk-averse behavior that prevented high scores. Participants familiarized through direct interaction were able to infer much of the same information through observation, and were more willing to take risks and experiment with different control modes, but reported weaker understanding of the agent's internal processes. Significant differences were seen between individual participants' risk tolerance and methods of AI interaction, which should be considered when designing human-AI control interfaces. Based on our findings, we recommend a human-AI team familiarization method that combines AI documentation, structured in-situ training, and exploratory interaction.

Figures (7)

• Figure 1: Overview of the ISR game environment used in the study. In this environment, a human participant and AI teammate ("agent") collaborated to identify targets. Participants interacted with the environment by setting waypoints via mouse clicks and sending commands to the agent using a button panel. The dark blue aircraft is controlled directly by the participant. The teal aircraft is the agent, controlled by a heuristic policy. Dots represent search targets that are unknown (yellow), hostile (red), or neutral (purple). Hostile targets' weapon ranges are shown as red rings, which are yellow until they are identified.
• Figure 2: A vignette from the AI Understanding quiz. For this question, participants could select "The agent will fly toward the single target in the bottom right" or "the agent will fly toward the cluster of targets."
• Figure 3: Breakdown of time spent using each control mode by each group, split between rounds 1-2 (left) and rounds 3-4 (right).
• Figure 4: Comparison of each group's propensity to command the agent to identify targets (Target mode) vs. to identify weapons (Weapon mode). Lines are drawn between each group's mean value in each round.
• Figure 5: The percentage of team weapon IDs that were performed by the agent. Higher percentages suggest that the participant had a higher tolerance for risk to the agent than to themselves.