Modelando procesos cognitivos de la lectura natural con GPT-2

TL;DR

This work finds that models like Ngrams and LSTM networks can partially model Predictability when used as a co-variable to explain readers' eye movements by using GPT-2 based models.

Abstract

The advancement of the Natural Language Processing field has enabled the development of language models with a great capacity for generating text. In recent years, Neuroscience has been using these models to better understand cognitive processes. In previous studies, we found that models like Ngrams and LSTM networks can partially model Predictability when used as a co-variable to explain readers' eye movements. In the present work, we further this line of research by using GPT-2 based models. The results show that this architecture achieves better outcomes than its predecessors.