Large-scale cloze evaluation reveals that token prediction tasks are neither lexically nor semantically aligned

TL;DR

While large models trained for longer are typically better estimators of human productions, but they reliably under-estimate the probabilities of human responses, over-rank rare responses, under-rank top responses, and produce highly distinct semantic spaces are found.

Abstract

In this work we compare the generative behavior at the next token prediction level in several language models by comparing them to human productions in the cloze task. We find that while large models trained for longer are typically better estimators of human productions, but they reliably under-estimate the probabilities of human responses, over-rank rare responses, under-rank top responses, and produce highly distinct semantic spaces. Altogether, this work demonstrates in a tractable, interpretable domain that LM generations can not be used as replacements of or models of the cloze task.

Figures (7)

• Figure 1: Correlation between human cloze probabilities and language model probabilities showing clear non-linearity in correlation and massive under-estimation of next-word probabilities.
• Figure 2: Rank correlation between Pythia-160M and human responses. Language models over-rank rare human responses (above solid line) and under-rank probable responses (below solid line).
• Figure 3: Pythia-160M next word prediction recovery compared to human productions.
• Figure 4: Pythia-160M recovery by cloze response rank and proportion of recoveries. Note that no responses with rank 11 are recovered at an equal or higher rank.
• Figure 5: Spearman rank correlation between LM (Pythia 160m-deduped) and human responses as functions of model size and training budget. Both show positive correlation plateauing after a certain point, staying at $\rho<0.5$ out of a maximum of $1$.