Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Investigating the Timescales of Language Processing with EEG and Language Models

Davide Turco, Conor Houghton

TL;DR

This work analyzes the temporal dynamics of language processing by aligning word representations from a pre-trained transformer with EEG data via a Temporal Response Function (TRF). By comparing embeddings from the GPT-2 embedding layer ($l=0$) with a deep layer ($l=8$) and isolating part-of-speech (POS) information through Linear Discriminant Analysis (LDA), the study reveals layer-specific TRF patterns that distinguish lexical from compositional processing and shows POS-related neural influences at later time windows. The results demonstrate distinct spatial and temporal EEG signatures for different transformer representations and provide evidence that POS-derived features modulate neural activity in the $>200$ ms range. Overall, the work highlights EEG’s utility for probing language processing at fine timescales and helps bridge artificial language models with human neural responses.

Abstract

This study explores the temporal dynamics of language processing by examining the alignment between word representations from a pre-trained transformer-based language model, and EEG data. Using a Temporal Response Function (TRF) model, we investigate how neural activity corresponds to model representations across different layers, revealing insights into the interaction between artificial language models and brain responses during language comprehension. Our analysis reveals patterns in TRFs from distinct layers, highlighting varying contributions to lexical and compositional processing. Additionally, we used linear discriminant analysis (LDA) to isolate part-of-speech (POS) representations, offering insights into their influence on neural responses and the underlying mechanisms of syntactic processing. These findings underscore EEG's utility for probing language processing dynamics with high temporal resolution. By bridging artificial language models and neural activity, this study advances our understanding of their interaction at fine timescales.

Investigating the Timescales of Language Processing with EEG and Language Models

TL;DR

This work analyzes the temporal dynamics of language processing by aligning word representations from a pre-trained transformer with EEG data via a Temporal Response Function (TRF). By comparing embeddings from the GPT-2 embedding layer () with a deep layer () and isolating part-of-speech (POS) information through Linear Discriminant Analysis (LDA), the study reveals layer-specific TRF patterns that distinguish lexical from compositional processing and shows POS-related neural influences at later time windows. The results demonstrate distinct spatial and temporal EEG signatures for different transformer representations and provide evidence that POS-derived features modulate neural activity in the ms range. Overall, the work highlights EEG’s utility for probing language processing at fine timescales and helps bridge artificial language models with human neural responses.

Abstract

This study explores the temporal dynamics of language processing by examining the alignment between word representations from a pre-trained transformer-based language model, and EEG data. Using a Temporal Response Function (TRF) model, we investigate how neural activity corresponds to model representations across different layers, revealing insights into the interaction between artificial language models and brain responses during language comprehension. Our analysis reveals patterns in TRFs from distinct layers, highlighting varying contributions to lexical and compositional processing. Additionally, we used linear discriminant analysis (LDA) to isolate part-of-speech (POS) representations, offering insights into their influence on neural responses and the underlying mechanisms of syntactic processing. These findings underscore EEG's utility for probing language processing dynamics with high temporal resolution. By bridging artificial language models and neural activity, this study advances our understanding of their interaction at fine timescales.
Paper Structure (10 sections, 1 equation, 1 figure)

This paper contains 10 sections, 1 equation, 1 figure.

Table of Contents

  1. Abstract
  2. Introduction
  3. Methodology
  4. Results
  5. Discussion
  6. Acknowledgements
  7. Supplementary material
  8. Glossary
  9. Additional methodology and implementation details
  10. Interpreting the TRF

Figures (1)

  • Figure 1: A) TRF obtained using the embedding layer (l=0). B) TRF obtained using a deep layer (l=8). C) Topographic map showing correlations between original and reconstructed EEG signal for a subject with high comprehension score. D) LDA-reduced representation space, with samples coloured by POS tag. E) TRF obtained using LDA-reduced representations.