Multipath parsing in the brain

TL;DR

The paper investigates whether human sentence comprehension maintains multiple syntactic analyses in parallel (multipath parsing) rather than a single path. It introduces an incremental generative dependency parser, enhanced with BLOOM-based encodings and Pfeiffer adapters, and compares single-path ($k=1$) to multipath ($k=5$) surprisal predictions against fMRI data collected while English and Chinese speakers listen to an audiobook. Across languages, syntactic surprisal derived from the five-path model better accounts for brain activity, with robust effects localized to bilateral superior temporal gyrus and related regions, supporting a multipath account of real-time parsing. This work advances neurocognitive modeling by linking beam-like, multi-hypothesis parsing to distributed brain responses, offering cross-linguistic evidence and suggesting avenues for higher-temporal-resolution studies and broader stimulus contexts.

Abstract

Humans understand sentences word-by-word, in the order that they hear them. This incrementality entails resolving temporary ambiguities about syntactic relationships. We investigate how humans process these syntactic ambiguities by correlating predictions from incremental generative dependency parsers with timecourse data from people undergoing functional neuroimaging while listening to an audiobook. In particular, we compare competing hypotheses regarding the number of developing syntactic analyses in play during word-by-word comprehension: one vs more than one. This comparison involves evaluating syntactic surprisal from a state-of-the-art dependency parser with LLM-adapted encodings against an existing fMRI dataset. In both English and Chinese data, we find evidence for multipath parsing. Brain regions associated with this multipath effect include bilateral superior temporal gyrus.

Figures (13)

• Figure 1: A sentence fragment from our stimulus text showing temporary syntactic ambiguity about the correct relationship between The and desert, which is resolved by hearing the next word (blue).
• Figure 2: Pipeline Overview: Pretokenized text from the treebank corpora is pre-processed by adding a leading space to each token. The pre-trained BLOOM model is fine-tuned using the Pfeiffer Adapter, and an additional single-layer feedforward neural network completes the encoding step. The dependency parser takes tuples of sequence encodings (stack ($\sigma$), buffer ($\beta$)) as input, and its system of classifiers is trained from random weights. Note that the word prediction classifier predicts over the size of the training data vocabulary + Berkeley unknown tokens instead of the BLOOM vocabulary. The final dependency tree output is evaluated in Table \ref{['tab:parser_accuracy']}. After training, we use the parser's transition probabilities (see Table \ref{['tab:transitions']}) to calculate syntactic surprisal (Section \ref{['sec:complexitymetric']}), which is added to other regressors (see Section \ref{['sec:Statistical']}), in a General Linear Model (GLM) which predicts BOLD signal (Section \ref{['sec:results']}).
• Figure 3: Labeled Accuracy Score (LAS), Unlabeled Accuracy Score (UAS), Label Accuracy for our parser trained and evaluated (dev set) on the Universal Dependencies ParTUT corpus. Each model uses a different BLOOM layer as a sequence encoder.
• Figure 4: A sentence from The Little Prince, showing incremental surprisal from joint probability decomposed into lexical (red), and syntactic surprisal (blue)
• Figure 5: A sentence fragment from The Little Prince showing shift-reduce parser actions with associated probabilities. Word generation probabilities are not shown here, but would be included in the full path calculation. In this example, the syntactic surprisal at time t = 3 is -log$_2$(0.55/1) = 0.86 for $k$=1, and for $k$=2 the top two paths are added: -log((0.55+0.45)/1) = 0.