Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Modeling Bilingual Sentence Processing: Evaluating RNN and Transformer Architectures for Cross-Language Structural Priming

Demi Zhang, Bushi Xiao, Chao Gao, Sangpil Youm, Bonnie J Dorr

TL;DR

It is indicated that transformers outperform RNNs in generating primed sentence structures, with accuracy rates that exceed 25.84% to 33.

Abstract

This study evaluates the performance of Recurrent Neural Network (RNN) and Transformer models in replicating cross-language structural priming, a key indicator of abstract grammatical representations in human language processing. Focusing on Chinese-English priming, which involves two typologically distinct languages, we examine how these models handle the robust phenomenon of structural priming, where exposure to a particular sentence structure increases the likelihood of selecting a similar structure subsequently. Our findings indicate that transformers outperform RNNs in generating primed sentence structures, with accuracy rates that exceed 25.84\% to 33. 33\%. This challenges the conventional belief that human sentence processing primarily involves recurrent and immediate processing and suggests a role for cue-based retrieval mechanisms. This work contributes to our understanding of how computational models may reflect human cognitive processes across diverse language families.

Modeling Bilingual Sentence Processing: Evaluating RNN and Transformer Architectures for Cross-Language Structural Priming

TL;DR

It is indicated that transformers outperform RNNs in generating primed sentence structures, with accuracy rates that exceed 25.84% to 33.

Abstract

This study evaluates the performance of Recurrent Neural Network (RNN) and Transformer models in replicating cross-language structural priming, a key indicator of abstract grammatical representations in human language processing. Focusing on Chinese-English priming, which involves two typologically distinct languages, we examine how these models handle the robust phenomenon of structural priming, where exposure to a particular sentence structure increases the likelihood of selecting a similar structure subsequently. Our findings indicate that transformers outperform RNNs in generating primed sentence structures, with accuracy rates that exceed 25.84\% to 33. 33\%. This challenges the conventional belief that human sentence processing primarily involves recurrent and immediate processing and suggests a role for cue-based retrieval mechanisms. This work contributes to our understanding of how computational models may reflect human cognitive processes across diverse language families.
Paper Structure (13 sections, 9 equations, 7 figures)

This paper contains 13 sections, 9 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Current Study
  4. Data Preparation
  5. Language Models
  6. Multi-head Attention in Transformer
  7. GRU Encoder and GRU Decoder
  8. Experimental Setup
  9. Results and Analyses
  10. Structural Priming Performance
  11. Crosslingual Structural Priming Effect
  12. Summary and Conclusions
  13. Future Directions

Figures (7)

  • Figure 1: Cross-language structure priming of human participant: C denotes Chinese, E denotes English.
  • Figure 2: Example of Active, Passive, Propositional Object (PO), and Double Object (DO). White highlighted sentence is original Chinese sentence, and yellow highlighted Sentence is word-to-word mapping between Chinese and English.
  • Figure 3: Example of test phase and evaluation process.
  • Figure 4: The workflow of the study includes PO (Propositional Object), DO (Double Object), Ac (Active), and Pa (Passive). In the training phase, raw bilingual data are preprocessed to generate token pairs. In the experiment phase, we employ transformer and RNN-based encoder-decoder architectures. In the testing phase, we evaluate the model's performance across four sentence structures using the BLEU metric.
  • Figure 5: BLEU Score for standard structural priming. Comparison of ground truth datasets for testing and calibration.
  • ...and 2 more figures