Mutual Theory of Mind in Human-AI Collaboration: An Empirical Study with LLM-driven AI Agents in a Real-time Shared Workspace Task

TL;DR

The results show that bidirectional communication leads to lower HAT performance, and the results' implications for designing AI agents that collaborate with humans in real-time shared workspace tasks are discussed.

Abstract

Theory of Mind (ToM) significantly impacts human collaboration and communication as a crucial capability to understand others. When AI agents with ToM capability collaborate with humans, Mutual Theory of Mind (MToM) arises in such human-AI teams (HATs). The MToM process, which involves interactive communication and ToM-based strategy adjustment, affects the team's performance and collaboration process. To explore the MToM process, we conducted a mixed-design experiment using a large language model-driven AI agent with ToM and communication modules in a real-time shared-workspace task. We find that the agent's ToM capability does not significantly impact team performance but enhances human understanding of the agent and the feeling of being understood. Most participants in our study believe verbal communication increases human burden, and the results show that bidirectional communication leads to lower HAT performance. We discuss the results' implications for designing AI agents that collaborate with humans in real-time shared workspace tasks.

Figures (13)

• Figure 1: Game Layout and Communication System of the Task.
• Figure 2: Game Mechanism. (a), (b), and (c) are the rules for preparing and serving burgers. (d) demonstrates the mechanism of overcooked beef and the rules for handling the fire caused by overcooked beef.
• Figure 3: Agent Framework. The framework shows how the LLM-driven agent with Theory of Mind and communication capability takes action and sends messages to the human player. We use a history buffer to save the game history, including game state, actions, and player messages. The Theory of Mind module uses history as input to analyze human behavior. The Policy and Message modules also have the history input to understand the whole picture of the game. We summarize the process of generating action and message: (1) The Theory of Mind module analyzes human behavior and messages, then generates beliefs about humans and provides a guide for adjusting the strategy for better team coordination and communication. (2) The Policy module uses the belief from the Theory of Mind module and the history to improve the agent's strategy by continually updating behavior guidelines. It outputs an action to control the agent. (3) The Message module uses the history, the inferred belief from the Theory of Mind module, and the guidelines from the Policy module to generate the message that aligns with the agents' actions and intentions.
• Figure 4: An example of Belief Output from the Theory of Mind Module.
• Figure 5: Prompt for Theory of Mind Module.