Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs

Abstract

Research on mental state reasoning in language models (LMs) has the potential to inform theories of human social cognition--such as the theory that mental state reasoning emerges in part from language exposure--and our understanding of LMs themselves. Yet much published work on LMs relies on a relatively small sample of closed-source LMs, limiting our ability to rigorously test psychological theories and evaluate LM capacities. Here, we replicate and extend published work on the false belief task by assessing LM mental state reasoning behavior across 41 open-weight models (from distinct model families). We find sensitivity to implied knowledge states in 34% of the LMs tested; however, consistent with prior work, none fully ``explain away'' the effect in humans. Larger LMs show increased sensitivity and also exhibit higher psychometric predictive power. Finally, we use LM behavior to generate and test a novel hypothesis about human cognition: both humans and LMs show a bias towards attributing false beliefs when knowledge states are cued using a non-factive verb (``John thinks...'') than when cued indirectly (``John looks in the...''). Unlike the primary effect of knowledge states, where human sensitivity exceeds that of LMs, the magnitude of the human knowledge cue effect falls squarely within the distribution of LM effect sizes-suggesting that distributional statistics of language can in principle account for the latter but not the former in humans. These results demonstrate the value of using larger samples of open-weight LMs to test theories of human cognition and evaluate LM capacities.

Figures (7)

• Figure 1: Distribution of Log Odds by condition (True Belief vs. False Belief) for the best-performing models, along with the by-item human baseline. Human Log Odds were calculated by comparing the proportion of Start vs. End responses for each item across participants.
• Figure 2: Accuracy on the false belief task by number of parameters. Overall, larger models performed better, though no model attained or surpassed the human baseline. Dashed red line corresponds to average human performance.
• Figure 3: Although no LM fully "explained away" the effect of Knowledge State in humans, some LMs did produce behavior that was more correlated with human behavior overall.
• Figure 4: Both LMs and humans exhibited a bias towards the Start (incorrect) location when a non-factive verb ("thinks") was used in the probe sentence. Both populations were also more likely to respond with the Start location in the False Belief condition, but this effect was considerably larger in humans.
• Figure 5: An example stimulus, formatted for continuation mode (top), instruction mode (middle), and instruction mode in a question-answer format (bottom). The text in red remains the same, but changes position slightly in the question-answer format. The blue [MASK] is where the probe of the Start and End token occurs. Text in bold are tokens introduced through the chat template for instruction-tuned models. In this example, the OLMo chat template is shown. These tokens will look different depending on the specific chat template unique to each model. The underlined text is the question probe used only in the question-answer format.