Beyond Words: Evaluating and Bridging Epistemic Divergence in User-Agent Interaction via Theory of Mind

TL;DR

This work reframes interaction with LLMs as a Theory of Mind (ToM) problem, formalizing a mechanism to detect and resolve epistemic divergence between a user’s subjective belief $b$ and the true environment state $s^*$. It introduces SynchToM, a four-domain benchmark with a two-stage data pipeline that generates belief–profile–state scenarios and 10-turn interaction trajectories, enabling evaluation of ToM utility in practical tasks. Across 11 models, results show ToM performance is domain-dependent and that ground-truth ToM factors significantly boost task success, while misalignment causes resolution failures; ToM reasoning is shown to be transferable and trainable via trajectory data. The authors demonstrate functional enhancements through trajectory-based reinforcement learning with ToM tokens and a training-free multi-agent collaboration setup, underscoring the practical impact of ToM for more robust, user-aligned AI agents in real-world settings.

Abstract

Large Language Models (LLMs) have developed rapidly and are widely applied to both general-purpose and professional tasks to assist human users. However, they still struggle to comprehend and respond to the true user needs when intentions and instructions are imprecisely conveyed, leading to a divergence between subjective user believes and true environment states. Resolving this epistemic divergence requires Theory of Mind (ToM), yet existing ToM evaluations for LLMs primarily focus on isolated belief inference, overlooking its functional utility in real-world interaction. To this end, we formalize ToM for LLMs as a mechanism for epistemic divergence detection and resolution, and propose a benchmark, \benchname, to assess how models reconcile user beliefs and profiles in practice. Results across 11 leading models reveal a significant limitation to identify underlying cognitive gaps that impede task success. To bridge this gap, we further curate a trajectory-based ToM dataset linking belief tracking with task-related state inference. The model trained on this data via reinforcement learning shows consistent improvement in reasoning about user mental states, leading to enhanced downstream performance. Our work highlights the practical value of ToM as an essential interaction-level mechanism rather than as a standalone reasoning skill.

Figures (6)

• Figure 1: Overview of our formalized mechanism for Theory of Mind (ToM). The environment possesses a true state $s$ that governs its actual evolution. Due to partial observability, the user perceives limited observations ($o^u$), forming a subjective belief ($b$) influenced by personal profiles such as preferences and domain knowledge. The user develops an interaction trajectory based on $b$, which may deviate from the gold trajectory. The agent observes the environment, user-provided observations, and interaction trajectory to infer $b$, and applies ToM reasoning to bridge the epistemic gap between $b$ and $s$, enabling more precise user-agent interaction.
• Figure 2: The SynchToM data generation pipeline. It contains two primary stages: ToM Scenario Synthesis (Stage 1), where raw data (e.g. QA pairs, context) from diverse tasks is converted into ToM scenarios with cognitive gap, comprising user profiles, true latent states, user latent belief, initial observations, user instruction and root cause of the gap. Trajectory Construction (Stage 2) generates multi-turn interaction trajectories that simulate the evolution of user beliefs based on the defined scenario in Stage 1. A Quality Control module is employed for each stage to ensure high-fidelity synthetic data. It validates instance across five dimensions, triggering regeneration if validation fails. We provide an example that generate an instance using a QA pair and context information of the original task data. The elements of scenario is generated based on the raw data, and trajectory reflects the evolution of user belief.
• Figure 3: Gains of the Solution metric when conditioning on Ground Truth (GT) Belief, GT Profile, and Randomly Shuffled context, respectively. While providing accurate GT information consistently yields gains in final solution, models benefit differentially from Belief versus Profile. In contrast, randomly shuffled context (mismatched latent belief and user profile) leads to consistent performance degradation. These reveal the causal relationship between Belief, Profile and Solution. Detailed causal analysis are provided in Appendix \ref{['app:gt']}.
• Figure 4: MiniMax-M2.1 trends with varied input trajectory turns. All metrics get improved when given more trajectory turns.
• Figure 5: Robustness analysis comparing the base model, Qwen3-8B to our ToM-enhanced model across five independent runs. The title of each subplots indicate the corresponding metric and category. Error bars represent standard deviation. Our model demonstrates consistent gains in Solution (bottom) and Latent Belief (top), particularly in multi-turn settings, validating the effectiveness of our training method.

Theorems & Definitions (1)

• Definition 3.1: User Cognitive State