Dynamic-Attention-based EEG State Transition Modeling for Emotion Recognition

TL;DR

A Dynamic-Attention-based EEG State Transition (DAEST) modeling method to characterize EEG spatiotemporal dynamics and is optimized within a contrastive learning framework for cross-subject emotion recognition.

Abstract

Electroencephalogram (EEG)-based emotion decoding can objectively quantify people's emotional state and has broad application prospects in human-computer interaction and early detection of emotional disorders. Recently emerging deep learning architectures have significantly improved the performance of EEG emotion decoding. However, existing methods still fall short of fully capturing the complex spatiotemporal dynamics of neural signals, which are crucial for representing emotion processing. This study proposes a Dynamic-Attention-based EEG State Transition (DAEST) modeling method to characterize EEG spatiotemporal dynamics. The model extracts spatiotemporal components of EEG that represent multiple parallel neural processes and estimates dynamic attention weights on these components to capture transitions in brain states. The model is optimized within a contrastive learning framework for cross-subject emotion recognition. The proposed method achieved state-of-the-art performance on three publicly available datasets: FACED, SEED, and SEED-V. It achieved 75.4% accuracy in the binary classification of positive and negative emotions and 59.3% in nine-class discrete emotion classification on the FACED dataset, 88.1% in the three-class classification of positive, negative, and neutral emotions on the SEED dataset, and 73.6% in five-class discrete emotion classification on the SEED-V dataset. The learned EEG spatiotemporal patterns and dynamic transition properties offer valuable insights into neural dynamics underlying emotion processing.

Figures (6)

• Figure 1: The architecture of the dynamic-attention-based EEG state transition model.
• Figure 2: Confusion matrices on the FACED-9, SEED, and SEED-V classification tasks (without DyA module and with DyA module).
• Figure 3: The pipeline of interpretability analysis. The integrated gradient method is used to identify the most important dimension or feature for each emotion in the Multilayer Perceptron. The temporal convolution kernel (or temporal filter), the spatial transition convolution kernel (or spatial filters), and the attention weights that produce this feature are visualized. Fourier transform of the temporal filter is identified as its frequency responses. Spatial activations are defined as multiplications of spatial filters with their corresponding inputs' covariance.
• Figure 4: Visualization of the spatiotemporal dynamics for the most important dimension of each emotion on the FACED-9 task. The temporal filters (the first column) and their frequency response (the second column), spatial filters and spatial activations (the third column), and example segments of attention weights (the fourth column) for the most important dimensions are shown. The “+”/”-” symbols in the parentheses following the emotion category name indicate that an increase in that feature corresponds to a high/lower probability of the corresponding emotion. The "interval" noted below spatial activations refers to the time interval of dilations for the corresponding spatial transition convolution kernel.
• Figure 5: Correlation of feature contributions to each emotion on the FACED-9 task.