Linguistic properties and model scale in brain encoding: from small to compressed language models

TL;DR

The paper addresses the minimal model capacity required for brain-aligned language representations by comparing small language models, large language models, and compressed variants using naturalistic fMRI data. It combines encoding/decoding analyses with the FlashHolmes linguistic benchmark to assess both neural predictivity and linguistic competence. The main findings reveal that brain alignment saturates at around three billion parameters and remains robust under most post-training quantization and moderate pruning, while certain aggressive compression schemes (notably GPTQ) degrade alignment; moreover, linguistic competence and neural alignment can dissociate under compression. Practically, these results advocate using compact, efficiently compressed models as principled, brain-grounded probes for NeuroAI research and potentially more accessible foundations for brain–language studies.

Abstract

Recent work has shown that scaling large language models (LLMs) improves their alignment with human brain activity, yet it remains unclear what drives these gains and which representational properties are responsible. Although larger models often yield better task performance and brain alignment, they are increasingly difficult to analyze mechanistically. This raises a fundamental question: what is the minimal model capacity required to capture brain-relevant representations? To address this question, we systematically investigate how constraining model scale and numerical precision affects brain alignment. We compare full-precision LLMs, small language models (SLMs), and compressed variants (quantized and pruned) by predicting fMRI responses during naturalistic language comprehension. Across model families up to 14B parameters, we find that 3B SLMs achieve brain predictivity indistinguishable from larger LLMs, whereas 1B models degrade substantially, particularly in semantic language regions. Brain alignment is remarkably robust to compression: most quantization and pruning methods preserve neural predictivity, with GPTQ as a consistent exception. Linguistic probing reveals a dissociation between task performance and brain predictivity: compression degrades discourse, syntax, and morphology, yet brain predictivity remains largely unchanged. Overall, brain alignment saturates at modest model scales and is resilient to compression, challenging common assumptions about neural scaling and motivating compact models for brain-aligned language modeling.

Figures (32)

• Figure 1: Does linguistic competence drive brain–model alignment? Participants listened to naturalistic English narratives while fMRI responses were recorded. Language models such as large, small, and their compressed variants, processed the same transcripts, and their internal representations were mapped to brain activity to quantify alignment. In parallel, the models were evaluated on the FlashHolmes benchmark to measure linguistic competence across morphology, syntax, semantics, discourse, and reasoning. By jointly comparing brain alignment and task performance across model scale and compression, we test whether reductions in linguistic competence induced by compression systematically degrade brain predictivity.
• Figure 2: Qwen2.5, LLaMA, and DeepSeek-R1: Normalized brain alignment was computed by averaging across participants, layers, and voxels. Red: 14b, Blue: 7b, Green: 3b, Orange: 1.5b, Solid: full-precision SLMs/LLMs, Patterned: quantized models. * at a particular bar indicates that the model's prediction performance is significantly better than 1b/1.5b SLMs. The top row shows whole-brain normalized alignment, while the bottom row focuses on a language-selective ROI (IFG).
• Figure 3: Normalized brain alignment averaged across participants and voxels, using the best-performing layer for Qwen2.5 model. Red: 14b, Blue: 7b/8b, Green: 3b, Orange: 1.5b, Solid: full-precision SLMs/LLMs, Patterned: quantized models. * at a particular bar indicates that the model's prediction performance is significantly better than 1b/1.5b SLMs. Plots for other model familes and regions are in Figs. \ref{['fig:qwen_vem_language']} and \ref{['fig:llama_vem_language']} in Appendix \ref{['app:alignmentLangROIs']}.
• Figure 4: Qwen2.5: Percentage change in brain alignment across model scales and quantization methods, shown on the flattened cortical surface of a representative subject (subject-5). Blue, orange, and red voxels indicate regions of information loss ((a) LLMs $\rightarrow$ 3B SLMs, (b) 3B SLMs $\rightarrow$ 3B SLMs GPTQ, (c) 3B SLMs $\rightarrow$ 1.5B SLMs, respectively), (d) while green voxels highlight regions of improvement for 3B SLMs AWQ over 3B SLMs. White voxels denote regions with no change. Results for other participants for Qwen2.5 and LLaMA models are in Appendix \ref{['app:flatMaps']}.
• Figure 5: Tradeoff between normalized brain alignment and linguistic competence performance on FlashHolmes Tasks (Qwen and LLaMA Model Families). Blue: 7b/8b, Green: 3b, Orange: 1.5b.