Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Bi-Hemispheric Emotion Mapping through EEG: A Dual-Stream Neural Network Approach

David Freire-Obregón, Daniel Hernández-Sosa, Oliverio J. Santana, Javier Lorenzo-Navarro, Modesto Castrillón-Santana

TL;DR

This work tackles EEG-based emotion recognition in a VR context by introducing a Bi-Hemispheric two-stream recurrent network that processes left and right hemisphere EEG signals in parallel. The methodology combines Conv1D-based topological processing, followed by LSTM temporal modeling, with a final dense layer and softmax for six emotion classes; pre-processing includes mastoid re-referencing and band-limited filtering. On a dataset with 20 training and 10 validation subjects, the Bi-Hemispheric model outperforms baselines, achieving a validation accuracy of 28.9% and a test-set average of 22.78%, notably better for joy and sadness. A temporal analysis reveals that early and late stimulus intervals significantly contribute to accuracy, guiding more efficient emotion inference and highlighting the benefit of leveraging hemispheric dynamics for nuanced emotion decoding in immersive environments.

Abstract

Emotion classification through EEG signals plays a significant role in psychology, neuroscience, and human-computer interaction. This paper addresses the challenge of mapping human emotions using EEG data in the Mapping Human Emotions through EEG Signals FG24 competition. Subjects mimic the facial expressions of an avatar, displaying fear, joy, anger, sadness, disgust, and surprise in a VR setting. EEG data is captured using a multi-channel sensor system to discern brain activity patterns. We propose a novel two-stream neural network employing a Bi-Hemispheric approach for emotion inference, surpassing baseline methods and enhancing emotion recognition accuracy. Additionally, we conduct a temporal analysis revealing that specific signal intervals at the beginning and end of the emotion stimulus sequence contribute significantly to improve accuracy. Leveraging insights gained from this temporal analysis, our approach offers enhanced performance in capturing subtle variations in the states of emotions.

Towards Bi-Hemispheric Emotion Mapping through EEG: A Dual-Stream Neural Network Approach

TL;DR

This work tackles EEG-based emotion recognition in a VR context by introducing a Bi-Hemispheric two-stream recurrent network that processes left and right hemisphere EEG signals in parallel. The methodology combines Conv1D-based topological processing, followed by LSTM temporal modeling, with a final dense layer and softmax for six emotion classes; pre-processing includes mastoid re-referencing and band-limited filtering. On a dataset with 20 training and 10 validation subjects, the Bi-Hemispheric model outperforms baselines, achieving a validation accuracy of 28.9% and a test-set average of 22.78%, notably better for joy and sadness. A temporal analysis reveals that early and late stimulus intervals significantly contribute to accuracy, guiding more efficient emotion inference and highlighting the benefit of leveraging hemispheric dynamics for nuanced emotion decoding in immersive environments.

Abstract

Emotion classification through EEG signals plays a significant role in psychology, neuroscience, and human-computer interaction. This paper addresses the challenge of mapping human emotions using EEG data in the Mapping Human Emotions through EEG Signals FG24 competition. Subjects mimic the facial expressions of an avatar, displaying fear, joy, anger, sadness, disgust, and surprise in a VR setting. EEG data is captured using a multi-channel sensor system to discern brain activity patterns. We propose a novel two-stream neural network employing a Bi-Hemispheric approach for emotion inference, surpassing baseline methods and enhancing emotion recognition accuracy. Additionally, we conduct a temporal analysis revealing that specific signal intervals at the beginning and end of the emotion stimulus sequence contribute significantly to improve accuracy. Leveraging insights gained from this temporal analysis, our approach offers enhanced performance in capturing subtle variations in the states of emotions.
Paper Structure (7 sections, 6 equations, 3 figures, 1 table)

This paper contains 7 sections, 6 equations, 3 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. Related work
  3. Methodology
  4. Experimental Setup
  5. Experimental Evaluation
  6. Temporal Analysis
  7. CONCLUSIONS

Figures (3)

  • Figure 1: Diagram illustrating the three-module stages for Bi-Hemispheric emotion recognition from EEG signals: Signal Pre-processing, Hemispheric Split, and Signal Classification using RNN.
  • Figure 2: Confusion matrix for the dual-branch approach on the validation set.
  • Figure 3: Temporal analysis of training and validation accuracies. Performance of Mono-Hemispheric and Bi-Hemispheric approaches across eight intervals (j0 to j7)