How Syntax Specialization Emerges in Language Models

TL;DR

The paper tackles how syntactic specialization emerges in transformer-based language models by introducing the Syntactic Sensitivity Index (SSI), an intrinsic measure that tracks layer- and neuron-level differentiation between grammatical and ungrammatical sentences across diverse syntactic phenomena without supervision. Using BLiMP as a minimal-pairs dataset, SSI is shown to correlate with syntactic task performance, identify functionally necessary high-SSI neurons via ablations, and reveal a developmental trajectory where specialization rises gradually, concentrates in certain layers, and exhibits a critical period around 16 million training tokens. The study further demonstrates initialization- and architecture-dependent dynamics, with larger models and more data amplifying abstraction and layer-localization, while different phenomena follow distinct acquisition timelines. These findings provide a mechanistic account of emergent syntax, linking internal representations to behavioral competence and offering practical resources (code and checkpoints) to advance future research into the internal dynamics of language models.

Abstract

Large language models (LLMs) have been found to develop surprising internal specializations: Individual neurons, attention heads, and circuits become selectively sensitive to syntactic structure, reflecting patterns observed in the human brain. While this specialization is well-documented, how it emerges during training and what influences its development remains largely unknown. In this work, we tap into the black box of specialization by tracking its formation over time. By quantifying internal syntactic consistency across minimal pairs from various syntactic phenomena, we identify a clear developmental trajectory: Syntactic sensitivity emerges gradually, concentrates in specific layers, and exhibits a 'critical period' of rapid internal specialization. This process is consistent across architectures and initialization parameters (e.g., random seeds), and is influenced by model scale and training data. We therefore reveal not only where syntax arises in LLMs but also how some models internalize it during training. To support future research, we will release the code, models, and training checkpoints upon acceptance.

Figures (12)

• Figure 1: (a) Left. Differences in SSI and grammatical judgment task accuracy between various checkpoints during training and the final model. The correlated downward trends suggest that increased syntactic specialization (lower $\Delta$SSI) is associated with improved grammatical accuracy (lower $\Delta$Accuracy). (b) Right. Human neural plasticity (orange) peaks early in development, consistent with the critical period for language acquisition Keshavan2014. Analogously, model syntactic specialization (blue), measured by SSI divergence across seeds (see \ref{['criticalperiod']}), emerges early in training and converges after approximately 16 million tokens.
• Figure 2: Specialized language related regions from fedorenko2024language. Broca’s area (red) is implicated in syntactic parsing and articulatory planning, while Wernicke’s area (light blue) supports speech sound processing. Additional regions, including the sensorimotor cortex (dark blue), primary auditory cortex (black), and the broader language network (purple), contribute to the perception, production, and comprehension of language.
• Figure 3: The neuron ablation result with 3 SD of original GPT-2 (left) and Pythia 2.8B (right).
• Figure 4: The developmental dynamics analysis. Each line represents a model checkpoint (from 0 to 2048M tokens, color-coded from dark blue to yellow), showing the SSI values across layers for a given training stage.
• Figure 5: Initialization Analysis. Each line represents a model initialized with a different seed. Models from the same family are shown in similar colours.