Mission: Impossible Language Models

TL;DR

The paper probes the boundary between possible and impossible languages by constructing synthetic, perturbation-based languages that range from English-like to counting-based, and evaluating GPT-2 small models trained on BabyLM. Using perplexity, surprisal, and causal-intervention analyses, the authors show that GPT-2 struggles more with the impossible languages than with natural English, challenging broad claims that LLMs are equally capable of learning all language types. The study demonstrates that model architecture and encoding choices influence learnability, and reveals modular, interpretable strategies the models employ to approximate counting-based grammars. Overall, the work advocates for using LLMs as comparative tools in cognitive and typological linguistics, while cautioning against overgeneralizing human-language learning capabilities from such models.

Abstract

Chomsky and others have very directly claimed that large language models (LLMs) are equally capable of learning languages that are possible and impossible for humans to learn. However, there is very little published experimental evidence to support such a claim. Here, we develop a set of synthetic impossible languages of differing complexity, each designed by systematically altering English data with unnatural word orders and grammar rules. These languages lie on an impossibility continuum: at one end are languages that are inherently impossible, such as random and irreversible shuffles of English words, and on the other, languages that may not be intuitively impossible but are often considered so in linguistics, particularly those with rules based on counting word positions. We report on a wide range of evaluations to assess the capacity of GPT-2 small models to learn these uncontroversially impossible languages, and crucially, we perform these assessments at various stages throughout training to compare the learning process for each language. Our core finding is that GPT-2 struggles to learn impossible languages when compared to English as a control, challenging the core claim. More importantly, we hope our approach opens up a productive line of inquiry in which different LLM architectures are tested on a variety of impossible languages in an effort to learn more about how LLMs can be used as tools for these cognitive and typological investigations.

Figures (15)

• Figure 1: Partial impossibility continuum of languages based on complexity. We assess the learnability of languages at different points in the continuum and push the (currently unclear) boundary between possible and impossible.
• Figure 2: Perplexities on a sample of 10K test sentences for each impossible language model over training steps. Error bars indicate 95% confidence intervals across 5 training runs initialized with different random seeds and evaluated on different test samples.
• Figure 3: Surprisal tests for each *Hop model over training steps. Error bars indicate 95% confidence intervals across 5 training runs initialized with different random seeds and evaluated on different test samples.
• Figure 4: An interchange intervention on the NoHop model with base input $b = \texttt{The man be}$ and source input $s = \texttt{The men be}$. The intervention is performed at the second layer and second token position, causing a change in prediction from S to P .
• Figure 5: Subject--verb agreement interchange intervention accuracies (IIA) for each *Hop model over training steps. Vertical axes denote the GPT-2 layer of the intervention, and horizontal axes denote the token position of the intervention. $t_d$, $t_s$, and $t_v$ represent the tokens for the determiner, subject, and verb, respectively. $t_1 \dots t_4$ represent the four tokens/words between the verb and its marker for TokenHop and WordHop. IIA values are averaged over results from 5 models initialized on different random seeds. See \ref{['sec:intervention_confidence_intervals']} for confidence intervals.