Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MASA-TCN: Multi-anchor Space-aware Temporal Convolutional Neural Networks for Continuous and Discrete EEG Emotion Recognition

Yi Ding, Su Zhang, Chuangao Tang, Cuntai Guan

TL;DR

This paper introduces MASA-TCN, a novel unified model that leverages the spatial learning capabilities of Temporal Convolutional Networks (TCNs) for EEG emotion regression and classification tasks and designs a multi-anchor block with attentive fusion, enabling the model to adaptively learn dynamic temporal dependencies within the EEG signals.

Abstract

Emotion recognition using electroencephalogram (EEG) mainly has two scenarios: classification of the discrete labels and regression of the continuously tagged labels. Although many algorithms were proposed for classification tasks, there are only a few methods for regression tasks. For emotion regression, the label is continuous in time. A natural method is to learn the temporal dynamic patterns. In previous studies, long short-term memory (LSTM) and temporal convolutional neural networks (TCN) were utilized to learn the temporal contextual information from feature vectors of EEG. However, the spatial patterns of EEG were not effectively extracted. To enable the spatial learning ability of TCN towards better regression and classification performances, we propose a novel unified model, named MASA-TCN, for EEG emotion regression and classification tasks. The space-aware temporal layer enables TCN to additionally learn from spatial relations among EEG electrodes. Besides, a novel multi-anchor block with attentive fusion is proposed to learn dynamic temporal dependencies. Experiments on two publicly available datasets show MASA-TCN achieves higher results than the state-of-the-art methods for both EEG emotion regression and classification tasks. The code is available at https://github.com/yi-ding-cs/MASA-TCN.

MASA-TCN: Multi-anchor Space-aware Temporal Convolutional Neural Networks for Continuous and Discrete EEG Emotion Recognition

TL;DR

This paper introduces MASA-TCN, a novel unified model that leverages the spatial learning capabilities of Temporal Convolutional Networks (TCNs) for EEG emotion regression and classification tasks and designs a multi-anchor block with attentive fusion, enabling the model to adaptively learn dynamic temporal dependencies within the EEG signals.

Abstract

Emotion recognition using electroencephalogram (EEG) mainly has two scenarios: classification of the discrete labels and regression of the continuously tagged labels. Although many algorithms were proposed for classification tasks, there are only a few methods for regression tasks. For emotion regression, the label is continuous in time. A natural method is to learn the temporal dynamic patterns. In previous studies, long short-term memory (LSTM) and temporal convolutional neural networks (TCN) were utilized to learn the temporal contextual information from feature vectors of EEG. However, the spatial patterns of EEG were not effectively extracted. To enable the spatial learning ability of TCN towards better regression and classification performances, we propose a novel unified model, named MASA-TCN, for EEG emotion regression and classification tasks. The space-aware temporal layer enables TCN to additionally learn from spatial relations among EEG electrodes. Besides, a novel multi-anchor block with attentive fusion is proposed to learn dynamic temporal dependencies. Experiments on two publicly available datasets show MASA-TCN achieves higher results than the state-of-the-art methods for both EEG emotion regression and classification tasks. The code is available at https://github.com/yi-ding-cs/MASA-TCN.
Paper Structure (29 sections, 18 equations, 4 figures, 10 tables)

This paper contains 29 sections, 18 equations, 4 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem formulation
  4. Neural networks for temporal pattern recognition
  5. Method
  6. Input construction
  7. Space-aware temporal convolutional layer
  8. Multi-anchor attentive fusion block
  9. Temporal convolutional layer
  10. Output layer for regression and classification tasks
  11. Experiments
  12. Datasets
  13. Preprocessing
  14. Evaluation metrics
  15. Experiment settings
  16. ...and 14 more sections

Figures (4)

  • Figure 1: The architecture of our MASA-TCN. There are four main parts of MASA-TCN: feature extraction block, MAAF block, TCN block, and regression/classification block. A sequence of five four-EEG-channel sub-segments is utilized as an example. The k of SAT refers to the length of the kernel in the temporal dimension. Best viewed in color.
  • Figure 2: Space-aware temporal convolutional layer. The SAT has two types of convolutional kernels: context kernels that extract the spectral patterns channel by channel and spatial fusion kernels that learn spatial patterns across all the channels. A context kernel of size (4, 3) is utilized for example. And a four-EEG-channel sample with four spectral features in each EEG channel is used as the example. Zero padding is added to make the context kernel a causal kernel along temporal dimension. Only one kernel for each type of CNN kernels is shown in the diagram for better view, the final output (on the top) consists of the outputs from more kernels (4 is utilized as the number of kernels of each type for demonstration purpose). Ch-$n$ represents $n$-th EEG channel, and $f_{n}$ is the $n$-th frequency band. Best viewed in color.
  • Figure 3: Effect of the depth and width of MASA-TCN.
  • Figure 4: Four representative samples of well and poorly regressed trials of MASA-TCN for CER. The y-axis is the valence score, and the x-axis is the index of the samples along temporal dimension.