Large Language Models as Theory of Mind Aware Generative Agents with Counterfactual Reflection

TL;DR

The paper introduces ToM-Agent, a theory-of-mind aware generative-agent architecture that disentangles beliefs from confidence to simulate first- and second-order ToM in open-domain dialogue. It combines a Self-BDI Aware module, a Vanilla BDI Tracking module, and a Counterfactual Reflection–based tracking module, augmented by foresight and reflection to iteratively update inferred BDIs from conversation history. Through empirical evaluation on empathetic and persuasive dialogue datasets using GPT-4 and GPT-3.5, the approach improves BDI inference and downstream dialogue metrics, with CR-based ToM delivering the strongest gains. The work advances open-domain social-skill modeling in large language models and offers insights for psychology and AI alignment by demonstrating how ToM reasoning can be instantiated in generative agents.

Abstract

Recent studies have increasingly demonstrated that large language models (LLMs) possess significant theory of mind (ToM) capabilities, showing the potential for simulating the tracking of mental states in generative agents. In this study, we propose a novel paradigm called ToM-agent, designed to empower LLMs-based generative agents to simulate ToM in open-domain conversational interactions. ToM-agent disentangles the confidence from mental states, facilitating the emulation of an agent's perception of its counterpart's mental states, such as beliefs, desires, and intentions (BDIs). Using past conversation history and verbal reflections, ToM-Agent can dynamically adjust counterparts' inferred BDIs, along with related confidence levels. We further put forth a counterfactual intervention method that reflects on the gap between the predicted responses of counterparts and their real utterances, thereby enhancing the efficiency of reflection. Leveraging empathetic and persuasion dialogue datasets, we assess the advantages of implementing the ToM-agent with downstream tasks, as well as its performance in both the first-order and the \textit{second-order} ToM. Our findings indicate that the ToM-agent can grasp the underlying reasons for their counterpart's behaviors beyond mere semantic-emotional supporting or decision-making based on common sense, providing new insights for studying large-scale LLMs-based simulation of human social behaviors.

Figures (4)

• Figure 1: Illustrations of proposed ToM-agent with BDIs tracking paradigm for LLMs-based generative agents aware theory of mind (ToM) and counterfactual reflection. As LLMs-based generative agents, two NPCs Bob and Alice are in conversational communication with each other. Bob generates his utterance based on the conversation history and his own beliefs, desires, and intentions (BDIs). Alice infers about Bob's top-k BDI candidates with confidence accordingly and predicts Bob's next-round response based on the conversation history and inferred BDIs. Then the counterfactual reflection is conducted based on the gap between the real response of Bob and the predicted response to make an updated plan, including add or delete manipulations for inferred top-k BDIs. Finally, Alice carries out the plan to update the inferred top-k BDIs of Bob along with the confidences accordingly.
• Figure 2: Illustrations of module components of ToM aware generative agents that could generate utterance according to self-BDI and tracking counterpart's BDI. Left Figure. Vanilla Counterpart's BDI Tracking Module. Middle Figure. Self-BDI-aware modules for generative agents. Right Figure. Counterpart's BDI tracking modules with counterfactual reflection.
• Figure 3: (a) Good Examples for BDI Infere. (b)Illustrations of curves of numerical changes in confidence for belief, desire, and intention in an episodic dialogue. As the dialogue progresses, the confidence values for belief, desire, and intention all increase steadily and eventually stabilize at high levels.
• Figure 4: Illustrations of curves of numerical changes in confidence for belief, desire, and intention in an episodic dialogue. Figure (a). As the dialogue progresses, the confidence values for Belief, Desire, and Intention all initially increase but eventually settle at the lower end of the scale. Figure (b). As the dialogue progresses, the confidence values for desire and intention increase steadily and eventually stabilize at high levels but belief eventually settles at the lower end of the scale.