Vectors from Larger Language Models Predict Human Reading Time and fMRI Data More Poorly when Dimensionality Expansion is Controlled

TL;DR

The paper investigates whether larger language models (LLMs) better predict human reading times and fMRI data. By using whole-vector representations from multiple LLM families and explicitly controlling for the number of predictors, the authors demonstrate that apparent gains with larger models largely reflect degrees of freedom rather than genuine alignment with human processing. Experiment 2 shows that even untrained models contribute to predictive power via DOF, and Experiment 3 reveals that training beyond DOF does not produce additional improvements for larger models. The findings challenge the notion of a straightforward quality-power relationship and suggest that improved model size does not necessarily translate to better cognitive-data fit, highlighting potential misalignment between LLM representations and human sentence processing.

Abstract

The impressive linguistic abilities of large language models (LLMs) have recommended them as models of human sentence processing, with some conjecturing a positive 'quality-power' relationship (Wilcox et al., 2023), in which language models' (LMs') fit to psychometric data continues to improve as their ability to predict words in context increases. This is important because it suggests that elements of LLM architecture, such as veridical attention to context and a unique objective of predicting upcoming words, reflect the architecture of the human sentence processing faculty, and that any inadequacies in predicting human reading time and brain imaging data may be attributed to insufficient model complexity, which recedes as larger models become available. Recent studies (Oh and Schuler, 2023) have shown this scaling inverts after a point, as LMs become excessively large and accurate, when word prediction probability (as information-theoretic surprisal) is used as a predictor. Other studies propose the use of entire vectors from differently sized LLMs, still showing positive scaling (Schrimpf et al., 2021), casting doubt on the value of surprisal as a predictor, but do not control for the larger number of predictors in vectors from larger LMs. This study evaluates LLM scaling using entire LLM vectors, while controlling for the larger number of predictors in vectors from larger LLMs. Results show that inverse scaling obtains, suggesting that inadequacies in predicting human reading time and brain imaging data may be due to substantial misalignment between LLMs and human sentence processing, which worsens as larger models are used.

Figures (3)

• Figure 1: Results from Experiment 1. Predictive power of vector elements from pre-trained LMs on human response data. Results of all datasets other than Pereira fMRI are the original correlation scores without normalization. For Pereira fMRI, following schrimpf2021neural (described in Section \ref{['exp1-reg-modeling']}), correlation scores were divided by the a ceiling value 0.32.
• Figure 2: Results from Experiment 2. (A) Predictive power of vector elements from Pythia models after 0 training steps (untrained). (B) Predictive power of vector elements from Pythia models after 143,000 training steps (fully trained). Results presented here followed the same procedure described in Section \ref{['exp1-reg-modeling']}. Correlation scores of all datasets other than Pereira fMRI are the original correlation scores without normalization; for Pereira fMRI, the correlation scores were normalized with a ceiling value 0.32.
• Figure 3: Results from Experiment 3. Contribution of additional 143,000 training steps to the predictive power of vector elements from fully trained Pythia models beyond the effect of degrees of freedom.