Frequency Explains the Inverse Correlation of Large Language Models' Size, Training Data Amount, and Surprisal's Fit to Reading Times

TL;DR

This study addresses why increasing Transformer LM size and training data degrades the alignment between surprisal estimates and human reading times. It combines four LM families and four naturalistic reading-time corpora to show that word frequency, especially for rare words, drives the inverse relation between size/data and fit, with larger models predicting rare words more accurately as training proceeds. A follow-up attribution analysis reveals that larger variants leverage effectively longer context windows and stronger local associations to predict rare words, explaining the deterioration in human-like reading-time fit. Overall, the work argues that superhumanly complex rare-word associations learned by large Transformers reduce their surprisal's cognitive plausibility, with implications for cognitive modeling and evaluation of language models.

Abstract

Recent studies have shown that as Transformer-based language models become larger and are trained on very large amounts of data, the fit of their surprisal estimates to naturalistic human reading times degrades. The current work presents a series of analyses showing that word frequency is a key explanatory factor underlying these two trends. First, residual errors from four language model families on four corpora show that the inverse correlation between model size and fit to reading times is the strongest on the subset of least frequent words, which is driven by excessively accurate predictions of larger model variants. Additionally, training dynamics reveal that during later training steps, all model variants learn to predict rare words and that larger model variants do so more accurately, which explains the detrimental effect of both training data amount and model size on fit to reading times. Finally, a feature attribution analysis demonstrates that larger model variants are able to accurately predict rare words based on both an effectively longer context window size as well as stronger local associations compared to smaller model variants. Taken together, these results indicate that Transformer-based language models' surprisal estimates diverge from human-like expectations due to the superhumanly complex associations they learn for predicting rare words.

Figures (14)

• Figure 1: Corpus-level perplexity measures from each GPT-2, GPT-Neo, OPT, and Pythia model variant, and mean squared errors of regression models that include each surprisal predictor on the four corpora of reading times. The ordered labels represent variants of different sizes, where '1' represents the smallest variant of each LM family.
• Figure 2: Corpus-level perplexity measures from each GPT-2, GPT-Neo, OPT, and Pythia model variant, and mean squared errors of regression models that include each surprisal predictor on the four corpora of reading times. The columns of subplots represent the five quintiles defined by unigram log-probabilities.
• Figure 3: Average surprisal values from each GPT-2, GPT-Neo, OPT, and Pythia model variant, and sum of squared errors of regression models that include each surprisal predictor on the four corpora of reading times. The columns of subplots represent the five quintiles defined by unigram log-probabilities. The top and bottom subplots of each row represent values from underpredicted and overpredicted data points, respectively.
• Figure 4: Proportion of surprisal values from select Pythia model variants on each quintile of the Dundee Corpus as a function of training steps. Proportions that are greater than .05 are annotated.
• Figure 5: SSEs from regression models containing surprisal predictors from select Pythia model variants on each quintile of the Dundee Corpus as a function of training steps. The columns of subplots share the scale but not the range of the x-axis for visual clarity.