Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models

TL;DR

This position paper argues that current ToM benchmarks for LLMs are incomplete and prone to data leakage and shortcuts. It introduces the ATOMS framework to taxonomy seven mental-state categories and analyzes existing benchmarks through this lens, revealing substantial gaps in coverage and situatedness. The authors advocate a situated ToM paradigm, where LLMs are treated as agents embedded in physical and social environments, demonstrated empirically via a grid-world pilot in MiniGrid. They propose concrete action items for benchmark design, evaluation protocols, and theoretical integration with language acquisition, aiming to advance LLMs as robust, interaction-ready ToM agents with broad applicability in AI systems.

Abstract

Large Language Models (LLMs) have generated considerable interest and debate regarding their potential emergence of Theory of Mind (ToM). Several recent inquiries reveal a lack of robust ToM in these models and pose a pressing demand to develop new benchmarks, as current ones primarily focus on different aspects of ToM and are prone to shortcuts and data leakage. In this position paper, we seek to answer two road-blocking questions: (1) How can we taxonomize a holistic landscape of machine ToM? (2) What is a more effective evaluation protocol for machine ToM? Following psychological studies, we taxonomize machine ToM into 7 mental state categories and delineate existing benchmarks to identify under-explored aspects of ToM. We argue for a holistic and situated evaluation of ToM to break ToM into individual components and treat LLMs as an agent who is physically situated in environments and socially situated in interactions with humans. Such situated evaluation provides a more comprehensive assessment of mental states and potentially mitigates the risk of shortcuts and data leakage. We further present a pilot study in a grid world setup as a proof of concept. We hope this position paper can facilitate future research to integrate ToM with LLMs and offer an intuitive means for researchers to better position their work in the landscape of ToM. Project page: https://github.com/Mars-tin/awesome-theory-of-mind

Figures (13)

• Figure 1: The ATOMS framework of beaudoin2020systematic, which identified 7 categories of mental states through meta-analysis of ToM studies for children.
• Figure 2: Number of available benchmarks for each mental state in ATOMS.
• Figure 3: A comparison of benchmark settings.
• Figure 4: An overview of the first and second order false belief task illustrated in a grid world setup. We simulate the unexpected transfer scenarios with two agents, and verbalize the environment and action traces to test if LLMs hold a correct understanding of the agents' false beliefs.
• Figure 5: An overview of the morally related emotional reaction tasks illustrated in a grid world setup. We simulate scenarios where an agent either directly witnesses or is ignorant of a morally related event, and verbalize the environment and action traces to test if LLMs hold a correct prediction of the agent's emotional reaction.