Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Source Domain Adaptation with Transformer-based Feature Generation for Subject-Independent EEG-based Emotion Recognition

Shadi Sartipi, Mujdat Cetin

TL;DR

The paper tackles cross-subject variability in EEG-based emotion recognition by introducing MSDA-TF, a framework that combines a CNN-based feature generator with a transformer encoder to learn discriminative features across multiple source subjects. It groups source subjects via Pearson correlation into multiple source domains and applies a moment-based alignment objective to minimize distributions differences between the unlabeled target and sources, as well as among the sources themselves. The approach yields higher cross-subject performance on the SEED dataset than several baselines, demonstrated by metrics such as accuracy and F1-score, and is supported by visualization of feature alignment. This method enhances subject-independent EEG emotion recognition, offering improved robustness for downstream BCI applications.

Abstract

Although deep learning-based algorithms have demonstrated excellent performance in automated emotion recognition via electroencephalogram (EEG) signals, variations across brain signal patterns of individuals can diminish the model's effectiveness when applied across different subjects. While transfer learning techniques have exhibited promising outcomes, they still encounter challenges related to inadequate feature representations and may overlook the fact that source subjects themselves can possess distinct characteristics. In this work, we propose a multi-source domain adaptation approach with a transformer-based feature generator (MSDA-TF) designed to leverage information from multiple sources. The proposed feature generator retains convolutional layers to capture shallow spatial, temporal, and spectral EEG data representations, while self-attention mechanisms extract global dependencies within these features. During the adaptation process, we group the source subjects based on correlation values and aim to align the moments of the target subject with each source as well as within the sources. MSDA-TF is validated on the SEED dataset and is shown to yield promising results.

Multi-Source Domain Adaptation with Transformer-based Feature Generation for Subject-Independent EEG-based Emotion Recognition

TL;DR

The paper tackles cross-subject variability in EEG-based emotion recognition by introducing MSDA-TF, a framework that combines a CNN-based feature generator with a transformer encoder to learn discriminative features across multiple source subjects. It groups source subjects via Pearson correlation into multiple source domains and applies a moment-based alignment objective to minimize distributions differences between the unlabeled target and sources, as well as among the sources themselves. The approach yields higher cross-subject performance on the SEED dataset than several baselines, demonstrated by metrics such as accuracy and F1-score, and is supported by visualization of feature alignment. This method enhances subject-independent EEG emotion recognition, offering improved robustness for downstream BCI applications.

Abstract

Although deep learning-based algorithms have demonstrated excellent performance in automated emotion recognition via electroencephalogram (EEG) signals, variations across brain signal patterns of individuals can diminish the model's effectiveness when applied across different subjects. While transfer learning techniques have exhibited promising outcomes, they still encounter challenges related to inadequate feature representations and may overlook the fact that source subjects themselves can possess distinct characteristics. In this work, we propose a multi-source domain adaptation approach with a transformer-based feature generator (MSDA-TF) designed to leverage information from multiple sources. The proposed feature generator retains convolutional layers to capture shallow spatial, temporal, and spectral EEG data representations, while self-attention mechanisms extract global dependencies within these features. During the adaptation process, we group the source subjects based on correlation values and aim to align the moments of the target subject with each source as well as within the sources. MSDA-TF is validated on the SEED dataset and is shown to yield promising results.
Paper Structure (9 sections, 3 equations, 3 figures, 3 tables, 1 algorithm)

This paper contains 9 sections, 3 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Feature Generator
  4. Domain Adaptation
  5. Learning Process
  6. Experimental Study
  7. Dataset
  8. Results
  9. Conclusion

Figures (3)

  • Figure 1: Overview of the proposed MSDA-TF.
  • Figure 2: t-SNE visualization of the proposed method (a) before adaptation, (b) alignment of source one and target, (c) alignment of source two and target, and (d) alignment of both sources.
  • Figure 3: Confusion matrices for source-domain training with no adaptation (left), and proposed MSDA-TF approach (right).