Left-right asymmetry in predicting brain activity from LLMs' representations emerges with their formal linguistic competence

TL;DR

The study investigates why left-right brain predictivity asymmetry emerges as LLMs train, proposing that formal linguistic competence underlies this effect. Using fMRI data from English and French listeners and a range of benchmarks, the authors show that the left-hemisphere advantage co-emerges with improved performance on formal linguistic tasks (e.g., BLiMP, Zorro, grammar acceptability) and the generation of well-formed text, but not with arithmetic, Dyck languages, or world-knowledge reasoning (ARC, Hellaswag). This pattern generalizes across model families (OLMo-2 7B and Pythia variants) and languages (French), and extends to cerebellar involvement, suggesting a robust link between syntactic/pattern-knowledge competence and LLM-to-brain alignment. The findings imply that syntactic-pattern knowledge primarily drives the observed hemispheric lateralization in brain predictivity, with functional competence lagging behind and contributing later. These insights refine our understanding of how artificial language systems align with human neural processing and point to region-specific developmental trajectories for future exploration.

Abstract

When humans and large language models (LLMs) process the same text, activations in the LLMs correlate with brain activity measured, e.g., with functional magnetic resonance imaging (fMRI). Moreover, it has been shown that, as the training of an LLM progresses, the performance in predicting brain activity from its internal activations improves more in the left hemisphere than in the right one. The aim of the present work is to understand which kind of competence acquired by the LLMs underlies the emergence of this left-right asymmetry. Using the OLMo-2 7B language model at various training checkpoints and fMRI data from English participants, we compare the evolution of the left-right asymmetry in brain scores alongside performance on several benchmarks. We observe that the asymmetry co-emerges with the formal linguistic abilities of the LLM. These abilities are demonstrated in two ways: by the model's capacity to assign a higher probability to an acceptable sentence than to a grammatically unacceptable one within a minimal contrasting pair, or its ability to produce well-formed text. On the opposite, the left-right asymmetry does not correlate with the performance on arithmetic or Dyck language tasks; nor with text-based tasks involving world knowledge and reasoning. We generalize these results to another family of LLMs (Pythia) and another language, namely French. Our observations indicate that the left-right asymmetry in brain predictivity matches the progress in formal linguistic competence (knowledge of linguistic patterns).

Figures (12)

• Figure 1: Phase transitions during training: "minimal pairs" benchmarks. Each panel displays the left-right hemispheric asymmetry in brain scores (blue curve, repeated across panels) and the performance on a given test, as a function of the number of tokens seen during training (on a log scale). The left panels show the performance on the linguistic tests, BLiMP and Zorro, and the right panels show the performance on the non-linguistic tests, Arithmetic and Dyck. Model used: OLMo-2-1124-7B. Brain scores are computed on the 25% most reliable voxels (see Fig. \ref{['fig:olmo-2_acc']} for whole brain results). To help compare the transitions, the benchmarks curves were scaled along the y-axis to match the left-right asymmetry curve, by minimizing the mean absolute vertical distance.
• Figure 2: Left-right hemispheric asymmetry aligns with the acquisition of formal linguistic competence, but not with high-level language comprehension. Formal competence is assessed by automatically evaluating the linguistic acceptability of text generated at each training checkpoint. The Hellaswag and ARC benchmarks assess world knowledge and commonsense reasoning.
• Figure 3: Quantitative comparison of the evolution of the left-right hemispheric brain scores and the various performance trajectories. (Left) After fitting a sigmoid to the evolution of a given quantity, we plot the results on a $(x_0, \beta)$ plane, where $x_0$ is the location of the transition along the log(number of tokens) axis, and $\beta$ the slope of the change (Right) Euclidean distance between the location on the $(x_0, \beta)$ plane of each benchmark and left-right asymmetry.
• Figure 4: Generalization to other language models. Results from minimal pairs benchmarks on OLMo-2-1124-7B extend to Pythia-2.8b & 6.9b models. Panel (a) reproduces data shown in Fig. \ref{['fig:olmo-2_rv_acc']} but with all curves superimposed. Panels (b) and (c) show the results for the Pythia models (figures split by tasks are provided in supplementary Fig. \ref{['fig:pythia_rv_acc']}).
• Figure 5: The evolution of the left-right asymmetry in brain predictivity in French subjects follows the acquisition of formal competence in this language by the LLM. The model is OLMo-2-1124-7B again. The blue line represents the evolution of the left-right asymmetry computed with on a French average subject. For comparison purposes, the dotted blue line reproduces the left-right asymmetry previously observed in English and displayed in Fig. \ref{['fig:olmo-2_rv_acc']}. In the left panel, the red curve tracks performance on the fr-grammar benchmark which measures French formal linguistic competence; in the right panel, the red curve shows the performance on French Hellaswag which assesses functional competence. (Note: both tests are part of the FrenchBench evaluation benchmark faysse2025croissantllm). See supplementary Fig. \ref{['fig:olmo-2_rv_fr_whole']} for results on the whole brain volume.