Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Attention-aware semantic relevance predicting Chinese sentence reading

Kun Sun

TL;DR

The resulting "attention-aware" metrics of semantic relevance can more accurately predict fixation durations in Chinese reading tasks recorded in an eye-tracking corpus than those calculated by existing approaches.

Abstract

In recent years, several influential computational models and metrics have been proposed to predict how humans comprehend and process sentence. One particularly promising approach is contextual semantic similarity. Inspired by the attention algorithm in Transformer and human memory mechanisms, this study proposes an ``attention-aware'' approach for computing contextual semantic relevance. This new approach takes into account the different contributions of contextual parts and the expectation effect, allowing it to incorporate contextual information fully. The attention-aware approach also facilitates the simulation of existing reading models and evaluate them. The resulting ``attention-aware'' metrics of semantic relevance can more accurately predict fixation durations in Chinese reading tasks recorded in an eye-tracking corpus than those calculated by existing approaches. The study's findings further provide strong support for the presence of semantic preview benefits in Chinese naturalistic reading. Furthermore, the attention-aware metrics of semantic relevance, being memory-based, possess high interpretability from both linguistic and cognitive standpoints, making them a valuable computational tool for modeling eye-movements in reading and further gaining insight into the process of language comprehension. Our approach underscores the potential of these metrics to advance our comprehension of how humans understand and process language, ultimately leading to a better understanding of language comprehension and processing.

Attention-aware semantic relevance predicting Chinese sentence reading

TL;DR

The resulting "attention-aware" metrics of semantic relevance can more accurately predict fixation durations in Chinese reading tasks recorded in an eye-tracking corpus than those calculated by existing approaches.

Abstract

In recent years, several influential computational models and metrics have been proposed to predict how humans comprehend and process sentence. One particularly promising approach is contextual semantic similarity. Inspired by the attention algorithm in Transformer and human memory mechanisms, this study proposes an ``attention-aware'' approach for computing contextual semantic relevance. This new approach takes into account the different contributions of contextual parts and the expectation effect, allowing it to incorporate contextual information fully. The attention-aware approach also facilitates the simulation of existing reading models and evaluate them. The resulting ``attention-aware'' metrics of semantic relevance can more accurately predict fixation durations in Chinese reading tasks recorded in an eye-tracking corpus than those calculated by existing approaches. The study's findings further provide strong support for the presence of semantic preview benefits in Chinese naturalistic reading. Furthermore, the attention-aware metrics of semantic relevance, being memory-based, possess high interpretability from both linguistic and cognitive standpoints, making them a valuable computational tool for modeling eye-movements in reading and further gaining insight into the process of language comprehension. Our approach underscores the potential of these metrics to advance our comprehension of how humans understand and process language, ultimately leading to a better understanding of language comprehension and processing.
Paper Structure (20 sections, 6 figures, 4 tables)

This paper contains 20 sections, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Word stroke count in Chinese reading
  4. Semantic similarity based on word vectors in cognitive studies
  5. Attention mechanism and contextual information
  6. Reading models
  7. Materials and Methods
  8. Materials
  9. Methods
  10. Statistical methods
  11. Results
  12. Study 1: GAMMs with random slope
  13. Study 2: GAMMs with random smooth
  14. Discussion
  15. Memory-based advantages and parallel processing
  16. ...and 5 more sections

Figures (6)

  • Figure 2: Partial effects for various metrics of semantic relevance. The x-axis represents the metric, while the y-axis signifies the fixation duration. Each curve in one plot demonstrates the relationship between a predictor variable and the response variable. More pronounced slopes on these curves signify a more substantial influence of the predictor on fixation durations, whereas more gentle slopes imply a less significant effect. ( Note: n = 76549; both attention-aware semantic relevance (- weights) and attention-aware semantic relevance (+ weights) are attention-aware metrics; contextual semantic relevance (- expectation) is not a typical attention-aware metric. "log" signifies log-transformation.)
  • Figure 3: The memory capability of the attention-aware approach and positional weights
  • Figure 4: The Pearson correlations among various metrics of our interest
  • Figure : A. Attention types in Transformers
  • Figure : A. Attention types in Transformers
  • ...and 1 more figures