Hierarchical Frequency Tagging Probe (HFTP): A Unified Approach to Investigate Syntactic Structure Representations in Large Language Models and the Human Brain

TL;DR

The paper introduces the Hierarchical Frequency Tagging Probe (HFTP), a frequency-domain framework to probe syntactic structure representations in both six LLMs and human brain data. By extracting neuron- and region-level spectral markers at canonical syntactic frequencies ($S(m,b)$ and $S(m,b_r)$) and applying Representational Similarity Analysis (RSA) on Structure Representational Dissimilarity Matrices (SRDMs), HFTP enables direct cross-modal alignment of hierarchical syntax representations. Across six diverse LLMs, HFTP reveals model-specific distributions of sentence- and phrase-selective neurons and demonstrates a robust left-hemisphere alignment with brain activity, albeit with distinct cortical localization compared to the models. The study shows that newer model upgrades can either improve or reduce brain alignment (e.g., Gemma 2 vs Gemma; Llama 3.1 vs Llama 2), emphasizing that performance gains do not necessarily translate to more human-like syntactic processing. Overall, HFTP provides a principled bridge between computational linguistics and cognitive neuroscience, with implications for interpretability, model design, and cross-linguistic research on naturalistic text processing.

Abstract

Large Language Models (LLMs) demonstrate human-level or even superior language abilities, effectively modeling syntactic structures, yet the specific computational modules responsible remain unclear. A key question is whether LLM behavioral capabilities stem from mechanisms akin to those in the human brain. To address these questions, we introduce the Hierarchical Frequency Tagging Probe (HFTP), a tool that utilizes frequency-domain analysis to identify neuron-wise components of LLMs (e.g., individual Multilayer Perceptron (MLP) neurons) and cortical regions (via intracranial recordings) encoding syntactic structures. Our results show that models such as GPT-2, Gemma, Gemma 2, Llama 2, Llama 3.1, and GLM-4 process syntax in analogous layers, while the human brain relies on distinct cortical regions for different syntactic levels. Representational similarity analysis reveals a stronger alignment between LLM representations and the left hemisphere of the brain (dominant in language processing). Notably, upgraded models exhibit divergent trends: Gemma 2 shows greater brain similarity than Gemma, while Llama 3.1 shows less alignment with the brain compared to Llama 2. These findings offer new insights into the interpretability of LLM behavioral improvements, raising questions about whether these advancements are driven by human-like or non-human-like mechanisms, and establish HFTP as a valuable tool bridging computational linguistics and cognitive neuroscience. This project is available at https://github.com/LilTiger/HFTP.

Figures (20)

• Figure 1: A framework for Hierarchical Frequency Tagging Probes (HFTP) and an illustration of neurons involved in different levels of hierarchical linguistic processing in both LLMs and the human brain. A, hierarchical linguistic structure in English and Chinese including syllable, phrase, and sentence. B, hierarchical linguistic pattern ($1\;\mathrm{Hz}$: sentence feature, $2\;\mathrm{Hz}$: phrase feature) observed both in LLMs and C, human brain.
• Figure 2: Alignment pipeline between LLMs and the human brain. SRDMs are computed for exclusive sentence/phrase and sentence&phrase MLP neurons and brain channels by comparing cosine similarities across different conditions. Subsequently, RSA (using Spearman correlation) is applied to quantify the similarity between the two SRDMs, thereby assessing the correspondence between model and brain representations.
• Figure 3: Hierarchical frequency patterns of MLP neurons selectively represent sentence features, phrase features, shared features of both and non-sensitive feature (from left to the right). Here, "experiment" denotes the original corpus, while "random" indicates the randomized version. Shaded bands show $\pm 1\;\mathrm{s.e.m.}$ computed across 10 shuffled-input activation partitions. Significant peaks (*$p < 0.05$, FDR corrected) indicate amplitudes stronger than neighboring frequencies within $\pm 0.5\;\mathrm{Hz}$. "Normalized Amplitude" represents the curves and bands scaled to a range of 0 to 1.
• Figure 4: Statistics of exclusive sentence/phrase MLP neurons and sentence & phrase MLP neurons in each layer across six LLMs.
• Figure 5: Hierarchical frequency patterns of sEEG channels selectively represent sentence features, phrase features, shared features of both and non-sensitive feature (from left to the right). Here, "setence" denotes the original sentence corpus, while "random" indicates the randomized version. Shaded bands show $\pm 1\;\mathrm{s.e.m.}$ computed across channels. Significant peaks (*$p < 0.05$, FDR corrected) indicate amplitudes stronger than neighboring frequencies within $\pm 0.5\;\mathrm{Hz}$. "Normalized ITPC" represents the curves and bands scaled to a range of 0 to 1.

Theorems & Definitions (3)

• Definition 1: Significant MLP Neurons
• Definition 2: Sentence MLP Neurons and Phrase MLP Neurons
• Definition 3: Sentence channels and Phrase channels