Feature Fusion Based on Mutual-Cross-Attention Mechanism for EEG Emotion Recognition
Yimin Zhao, Jin Gu
TL;DR
This work tackles the challenge of accurate and interpretable EEG-based emotion recognition by introducing a purely mathematical Mutual-Cross-Attention (MCA) mechanism to fuse differential-entropy (DE) and power-spectral-density (PSD) features within a novel Channel-Frequency-Time 3D feature framework. A customized 3D-CNN processes the fused features, achieving high accuracy on the DEAP dataset (valence ≈ 99.499%, arousal ≈ 99.309%), with ablation studies confirming the complementary value of DE and PSD and the superiority of MCA over traditional fusion. The key contributions are the MCA fusion method, the Channel-Frequency-Time 3D feature design, and the specialized 3D-CNN architecture, all aimed at fast, interpretable emotion discrimination from EEG. The results suggest strong potential for clinical psychotherapy applications and motivate further exploration of transformer-based MCA in larger, more complex datasets.
Abstract
An objective and accurate emotion diagnostic reference is vital to psychologists, especially when dealing with patients who are difficult to communicate with for pathological reasons. Nevertheless, current systems based on Electroencephalography (EEG) data utilized for sentiment discrimination have some problems, including excessive model complexity, mediocre accuracy, and limited interpretability. Consequently, we propose a novel and effective feature fusion mechanism named Mutual-Cross-Attention (MCA). Combining with a specially customized 3D Convolutional Neural Network (3D-CNN), this purely mathematical mechanism adeptly discovers the complementary relationship between time-domain and frequency-domain features in EEG data. Furthermore, the new designed Channel-PSD-DE 3D feature also contributes to the high performance. The proposed method eventually achieves 99.49% (valence) and 99.30% (arousal) accuracy on DEAP dataset.