Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reasoning Promotes Robustness in Theory of Mind Tasks

Ian B. de Haan, Peter van der Putten, Max van Duijn

TL;DR

This work investigates whether RLVR-trained reasoning models truly expand theory-of-m mind abilities or primarily improve robustness to prompts in ToM tasks. By applying established ToM tests, novel prompt adaptations, and multiple benchmarks, the study finds that gains are largely due to more robust inference paths rather than novel ToM reasoning. Across psychological tests and literature benchmarks, reasoning models show markedly improved robustness to prompt variations, with some edge cases where limitations persist. The findings suggest a shift in how ToM performance should be interpreted in LLMs and highlight robustness as a key strength of RLVR-trained models. The work emphasizes careful benchmarking and prompts-aware evaluation to accurately assess social-cognitive capabilities in AI systems.

Abstract

Large language models (LLMs) have recently shown strong performance on Theory of Mind (ToM) tests, prompting debate about the nature and true performance of the underlying capabilities. At the same time, reasoning-oriented LLMs trained via reinforcement learning with verifiable rewards (RLVR) have achieved notable improvements across a range of benchmarks. This paper examines the behavior of such reasoning models in ToM tasks, using novel adaptations of machine psychological experiments and results from established benchmarks. We observe that reasoning models consistently exhibit increased robustness to prompt variations and task perturbations. Our analysis indicates that the observed gains are more plausibly attributed to increased robustness in finding the correct solution, rather than to fundamentally new forms of ToM reasoning. We discuss the implications of this interpretation for evaluating social-cognitive behavior in LLMs.

Reasoning Promotes Robustness in Theory of Mind Tasks

TL;DR

This work investigates whether RLVR-trained reasoning models truly expand theory-of-m mind abilities or primarily improve robustness to prompts in ToM tasks. By applying established ToM tests, novel prompt adaptations, and multiple benchmarks, the study finds that gains are largely due to more robust inference paths rather than novel ToM reasoning. Across psychological tests and literature benchmarks, reasoning models show markedly improved robustness to prompt variations, with some edge cases where limitations persist. The findings suggest a shift in how ToM performance should be interpreted in LLMs and highlight robustness as a key strength of RLVR-trained models. The work emphasizes careful benchmarking and prompts-aware evaluation to accurately assess social-cognitive capabilities in AI systems.

Abstract

Large language models (LLMs) have recently shown strong performance on Theory of Mind (ToM) tests, prompting debate about the nature and true performance of the underlying capabilities. At the same time, reasoning-oriented LLMs trained via reinforcement learning with verifiable rewards (RLVR) have achieved notable improvements across a range of benchmarks. This paper examines the behavior of such reasoning models in ToM tasks, using novel adaptations of machine psychological experiments and results from established benchmarks. We observe that reasoning models consistently exhibit increased robustness to prompt variations and task perturbations. Our analysis indicates that the observed gains are more plausibly attributed to increased robustness in finding the correct solution, rather than to fundamentally new forms of ToM reasoning. We discuss the implications of this interpretation for evaluating social-cognitive behavior in LLMs.
Paper Structure (41 sections, 2 figures, 6 tables)

This paper contains 41 sections, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Chains of Thought and Reasoning Models
  4. Theory of Mind and LLMs
  5. Ways of Improving Models' Performance
  6. Orders of ToM
  7. Methods
  8. Models
  9. Psychological tests
  10. Prompting
  11. Scoring
  12. Code
  13. Benchmarks
  14. Results
  15. Psychological Tests
  16. ...and 26 more sections

Figures (2)

  • Figure 1: Image illustrating the chain-of-thought prompting techniques (a) Traditional prompting, doesn't elicit CoT (b) Few-shot prompting, elicits CoT at the cost of a few examples (c) Zero shot prompting, elicits CoT for free
  • Figure 2: The prompting flows for the Sally-Anne tests for (a) Models with no reasoning, (b) Other reasoning models, and (c) Claude. GPT-5 prompting is a mix between (a) and (c) and omitted for brevity. Illustrations were elaborated with the help of Claude AI.