Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DuA: Dual Attentive Transformer in Long-Term Continuous EEG Emotion Analysis

Yue Pan, Qile Liu, Qing Liu, Li Zhang, Gan Huang, Xin Chen, Fali Li, Peng Xu, Zhen Liang

TL;DR

The paper addresses the challenge of long-term continuous EEG emotion analysis by proposing the Dual Attentive (DuA) transformer, which processes entire EEG trials (trial-based emotion analysis) using three modules: a spatial-spectral network, a temporal network, and a transfer learning component. It leverages an encoder-only transformer with a CLS token to capture long-range temporal dependencies while integrating spatial and spectral EEG features, and employs adversarial domain adaptation to improve cross-subject generalization. Empirical results on a self-constructed long-term EEG database and two benchmarks (SEED, SEED-IV) show substantial gains over both traditional and deep baselines under trial-based LOSO evaluation, including a reported average improvement of 5.28%. The approach advances practical affective brain-computer interfaces by enabling robust, adaptable emotion decoding across long durations and individuals, with potential impact on mental health monitoring and human-computer interaction.

Abstract

Affective brain-computer interfaces (aBCIs) are increasingly recognized for their potential in monitoring and interpreting emotional states through electroencephalography (EEG) signals. Current EEG-based emotion recognition methods perform well with short segments of EEG data. However, these methods encounter significant challenges in real-life scenarios where emotional states evolve over extended periods. To address this issue, we propose a Dual Attentive (DuA) transformer framework for long-term continuous EEG emotion analysis. Unlike segment-based approaches, the DuA transformer processes an entire EEG trial as a whole, identifying emotions at the trial level, referred to as trial-based emotion analysis. This framework is designed to adapt to varying signal lengths, providing a substantial advantage over traditional methods. The DuA transformer incorporates three key modules: the spatial-spectral network module, the temporal network module, and the transfer learning module. The spatial-spectral network module simultaneously captures spatial and spectral information from EEG signals, while the temporal network module detects temporal dependencies within long-term EEG data. The transfer learning module enhances the model's adaptability across different subjects and conditions. We extensively evaluate the DuA transformer using a self-constructed long-term EEG emotion database, along with two benchmark EEG emotion databases. On the basis of the trial-based leave-one-subject-out cross-subject cross-validation protocol, our experimental results demonstrate that the proposed DuA transformer significantly outperforms existing methods in long-term continuous EEG emotion analysis, with an average enhancement of 5.28%.

DuA: Dual Attentive Transformer in Long-Term Continuous EEG Emotion Analysis

TL;DR

The paper addresses the challenge of long-term continuous EEG emotion analysis by proposing the Dual Attentive (DuA) transformer, which processes entire EEG trials (trial-based emotion analysis) using three modules: a spatial-spectral network, a temporal network, and a transfer learning component. It leverages an encoder-only transformer with a CLS token to capture long-range temporal dependencies while integrating spatial and spectral EEG features, and employs adversarial domain adaptation to improve cross-subject generalization. Empirical results on a self-constructed long-term EEG database and two benchmarks (SEED, SEED-IV) show substantial gains over both traditional and deep baselines under trial-based LOSO evaluation, including a reported average improvement of 5.28%. The approach advances practical affective brain-computer interfaces by enabling robust, adaptable emotion decoding across long durations and individuals, with potential impact on mental health monitoring and human-computer interaction.

Abstract

Affective brain-computer interfaces (aBCIs) are increasingly recognized for their potential in monitoring and interpreting emotional states through electroencephalography (EEG) signals. Current EEG-based emotion recognition methods perform well with short segments of EEG data. However, these methods encounter significant challenges in real-life scenarios where emotional states evolve over extended periods. To address this issue, we propose a Dual Attentive (DuA) transformer framework for long-term continuous EEG emotion analysis. Unlike segment-based approaches, the DuA transformer processes an entire EEG trial as a whole, identifying emotions at the trial level, referred to as trial-based emotion analysis. This framework is designed to adapt to varying signal lengths, providing a substantial advantage over traditional methods. The DuA transformer incorporates three key modules: the spatial-spectral network module, the temporal network module, and the transfer learning module. The spatial-spectral network module simultaneously captures spatial and spectral information from EEG signals, while the temporal network module detects temporal dependencies within long-term EEG data. The transfer learning module enhances the model's adaptability across different subjects and conditions. We extensively evaluate the DuA transformer using a self-constructed long-term EEG emotion database, along with two benchmark EEG emotion databases. On the basis of the trial-based leave-one-subject-out cross-subject cross-validation protocol, our experimental results demonstrate that the proposed DuA transformer significantly outperforms existing methods in long-term continuous EEG emotion analysis, with an average enhancement of 5.28%.
Paper Structure (20 sections, 14 equations, 3 figures, 6 tables, 1 algorithm)

This paper contains 20 sections, 14 equations, 3 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Short Segments based EEG Analysis
  4. Transfer Learning in aBCI
  5. Transformer Network with Attention Mechanism
  6. Methodology
  7. Transformer Basics
  8. Spatial-Spectral Network Module
  9. Temporal Network Module
  10. Transfer Learning Module
  11. Experimental Results
  12. EEG Data and Experimental Protocol
  13. Implementation Details
  14. Emotion Recognition Results
  15. Discussion Conclusion
  16. ...and 5 more sections

Figures (3)

  • Figure 1: The overall architecture of the proposed Dual Attentive (DuA) transformer model. (a) the main pipeline of the model, and (b) details the structure of the spatial-spectral network module.
  • Figure 2: An illustration of different classification tasks.
  • Figure 3: Hyperparameter analysis of (a) the number of attention heads in the spatial-spectral network, (b) the number of layers in the spatial-spectral network, and (c) the hidden size in the temporal network.