Exploring Similarity between Neural and LLM Trajectories in Language Processing

TL;DR

This work addresses how brain activity during language comprehension relates to the evolving internal representations of large language models (LLMs). The authors quantify both representational similarity and dynamical trajectory alignment between EEG data and 16 public LLMs across English and Chinese, employing ridge regression with RSA/CKA and a Latent Trajectory Comparison framework that includes Magnitude, Angle, Uncertainty, Confidence, and MI, plus the Dynamic Representational Alignment metric. Key findings show that middle-to-high LLM layers contribute to semantic integration resembling the brain's N400 component, while brain activity remains continuous whereas LLMs exhibit discrete, stage-like bursts; multilingual alignment is stronger for English than Chinese, likely due to instruction-tuning data biases. The study provides a framework for assessing brain–LLM alignment beyond static representations, highlighting both shared semantic processing mechanisms and fundamental differences in temporal dynamics, which informs future multilingual model design and cognitive neuroscience research.

Abstract

Understanding the similarity between large language models (LLMs) and human brain activity is crucial for advancing both AI and cognitive neuroscience. In this study, we provide a multilinguistic, large-scale assessment of this similarity by systematically comparing 16 publicly available pretrained LLMs with human brain responses during natural language processing tasks in both English and Chinese. Specifically, we use ridge regression to assess the representational similarity between LLM embeddings and electroencephalography (EEG) signals, and analyze the similarity between the "neural trajectory" and the "LLM latent trajectory." This method captures key dynamic patterns, such as magnitude, angle, uncertainty, and confidence. Our findings highlight both similarities and crucial differences in processing strategies: (1) We show that middle-to-high layers of LLMs are central to semantic integration and correspond to the N400 component observed in EEG; (2) The brain exhibits continuous and iterative processing during reading, whereas LLMs often show discrete, stage-end bursts of activity, which suggests a stark contrast in their real-time semantic processing dynamics. This study could offer new insights into LLMs and neural processing, and also establish a critical framework for future investigations into the alignment between artificial intelligence and biological intelligence.

Figures (9)

• Figure 1: Comparison of human brain EEG signals and LLM internal states to explore similarities between human thought processes and model processing trajectories.
• Figure 2: Overview of the proposed methodology for investigating brain-LLM language processing similarities. (a) Framework for measuring Representation similarity: Pearson correlation (ridge regression), spatiotemporal (ST) alignment, and latent trajectory comparison (LTC). (b) LTC: Trajectories across layers and time are compared. (c) Magnitude and angular dynamics: Analysing intensity and directionality. (d) Uncertainty and confidence dynamics.
• Figure 3: Similarity analysis. (a) Visualization of EEG-LLM similarity via RDMs. (b) Comparison across different subjects. (c) Trend of similarity between LLM layers and EEG responses.
• Figure 4: Topographic maps and connectivity analysis. (a) Topographic maps of EEG–LLM correlations. (b) EEG functional connectivity patterns. (c) Functional connectivity predicted by LLM.
• Figure 5: Temporal and dynamic comparisons between EEGs and LLMs. (a) Magnitude dynamics, (b) Angle dynamics, (c) Uncertainty dynamics, (d) Confidence dynamics, (e) MI dynamics.