Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gap the (Theory of) Mind: Sharing Beliefs About Teammates' Goals Boosts Collaboration Perception, Not Performance

Yotam Amitai, Reuth Mirsky, Ofra Amir

TL;DR

The study investigates whether an AI agent’s sharing of its inferred human teammate’s goals improves collaboration and performance in ad-hoc human-agent teams. Using a tool-fetching domain with NR, VG, and VGod conditions, the authors assess objective performance, subjective satisfaction, cognitive load, and qualitative responses; they find no significant gains in objective metrics or satisfaction, though qualitative analyses reveal shifts toward strategic behavior and enhanced perceived collaboration. The results underscore a nuanced trade-off: information sharing can boost the user’s sense of partnership without reliably enhancing efficiency, and cognitive load dynamics may limit the benefits of transparency. The work highlights design implications for goal-sharing interfaces, emphasizing balance between informativeness and simplicity and identifying failure modes to guide future research on when transparency truly benefits collaboration.

Abstract

In human-agent teams, openly sharing goals is often assumed to enhance planning, collaboration, and effectiveness. However, direct communication of these goals is not always feasible, requiring teammates to infer their partner's intentions through actions. Building on this, we investigate whether an AI agent's ability to share its inferred understanding of a human teammate's goals can improve task performance and perceived collaboration. Through an experiment comparing three conditions-no recognition (NR), viable goals (VG), and viable goals on-demand (VGod) - we find that while goal-sharing information did not yield significant improvements in task performance or overall satisfaction scores, thematic analysis suggests that it supported strategic adaptations and subjective perceptions of collaboration. Cognitive load assessments revealed no additional burden across conditions, highlighting the challenge of balancing informativeness and simplicity in human-agent interactions. These findings highlight the nuanced trade-off of goal-sharing: while it fosters trust and enhances perceived collaboration, it can occasionally hinder objective performance gains.

Gap the (Theory of) Mind: Sharing Beliefs About Teammates' Goals Boosts Collaboration Perception, Not Performance

TL;DR

The study investigates whether an AI agent’s sharing of its inferred human teammate’s goals improves collaboration and performance in ad-hoc human-agent teams. Using a tool-fetching domain with NR, VG, and VGod conditions, the authors assess objective performance, subjective satisfaction, cognitive load, and qualitative responses; they find no significant gains in objective metrics or satisfaction, though qualitative analyses reveal shifts toward strategic behavior and enhanced perceived collaboration. The results underscore a nuanced trade-off: information sharing can boost the user’s sense of partnership without reliably enhancing efficiency, and cognitive load dynamics may limit the benefits of transparency. The work highlights design implications for goal-sharing interfaces, emphasizing balance between informativeness and simplicity and identifying failure modes to guide future research on when transparency truly benefits collaboration.

Abstract

In human-agent teams, openly sharing goals is often assumed to enhance planning, collaboration, and effectiveness. However, direct communication of these goals is not always feasible, requiring teammates to infer their partner's intentions through actions. Building on this, we investigate whether an AI agent's ability to share its inferred understanding of a human teammate's goals can improve task performance and perceived collaboration. Through an experiment comparing three conditions-no recognition (NR), viable goals (VG), and viable goals on-demand (VGod) - we find that while goal-sharing information did not yield significant improvements in task performance or overall satisfaction scores, thematic analysis suggests that it supported strategic adaptations and subjective perceptions of collaboration. Cognitive load assessments revealed no additional burden across conditions, highlighting the challenge of balancing informativeness and simplicity in human-agent interactions. These findings highlight the nuanced trade-off of goal-sharing: while it fosters trust and enhances perceived collaboration, it can occasionally hinder objective performance gains.
Paper Structure (34 sections, 3 figures)

This paper contains 34 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Information Sharing in Teamwork and Decision-Making
  4. Explanations in Human-Agent Collaboration
  5. Cognitive Load and Its Effects on Performance
  6. Distinction Between Perceived and Objective Help
  7. Experimental Methodology
  8. Domain
  9. Goal Recognition and Interaction
  10. Experiment Design
  11. Participants
  12. Task and Procedure
  13. Metrics Collected
  14. Data Analysis
  15. Results and Experiment Discussion
  16. ...and 19 more sections

Figures (3)

  • Figure 1: The Worker-Fetcher Environment. The worker (W) moves to a numbered goal station (1, 2, 3, 4). The fetcher (F) infers the goal, retrieves the corresponding tool (T), and delivers it to the worker.
  • Figure 2: Box plot distributions of task-related metrics across the experimental conditions: Duration (top-left), Performance (top-right), Need for Cognition (N4C) Score (bottom-left), and AI Attitude Score (bottom-right).
  • Figure 3: NASA-TLX scores across conditions, adjusted s.t. higher scores indicate better outcomes.