Emotion Classification from Multi-Channel EEG Signals Using HiSTN: A Hierarchical Graph-based Spatial-Temporal Approach

Abstract

This study introduces a parameter-efficient Hierarchical Spatial Temporal Network (HiSTN) specifically designed for the task of emotion classification using multi-channel electroencephalogram data. The network incorporates a graph hierarchy constructed from bottom-up at various abstraction levels, offering the dual advantages of enhanced task-relevant deep feature extraction and a lightweight design. The model's effectiveness is further amplified when used in conjunction with a proposed unique label smoothing method. Comprehensive benchmark experiments reveal that this combined approach yields high, balanced performance in terms of both quantitative and qualitative predictions. HiSTN, which has approximately 1,000 parameters, achieves mean F1 scores of 96.82% (valence) and 95.62% (arousal) in subject-dependent tests on the rarely-utilized 5-classification task problem from the DREAMER dataset. In the subject-independent settings, the same model yields mean F1 scores of 78.34% for valence and 81.59% for arousal. The adoption of the Sequential Top-2 Hit Rate (Seq2HR) metric highlights the significant enhancements in terms of the balance between model's quantitative and qualitative for predictions achieved through our approach when compared to training with regular one-hot labels. These improvements surpass 50% in subject-dependent tasks and 30% in subject-independent tasks. The study also includes relevant ablation studies and case explorations to further elucidate the workings of the proposed model and enhance its interpretability.

Figures (9)

• Figure 1: The design of the proposed HiSTN network. A closer look at the Hierarchy Core (enclosed by the red dashed line) is unpacked in Fig. \ref{['fig:node_fusion']}.
• Figure 2: Unpacking the node fusion block. FL: Frontal Left, FR: Frontal Right, PL: Parietal Left, PR: Parietal Right. At the intermediate stage, Region block R "summarizes" the learned information from previous node/channel level features. This processed information per region is then further summarized by the Global block G over the whole graph.
• Figure 3: 2D embedding of deep features using UMAP when different models are trained with (1) regular OneHot label encoding, and (2) our proposed special label-smoothing. Left: Valence prediction with date from Subject S3, Right: Valence prediction with data from Subject S23.
• Figure 4: Deep features extracted in the temporal view and the spatial view at a time snapshot. From top to bottom: channel level, region Level and graph Level. The spatial view for the graph level is not shown since it is a scalar value.
• Figure 5: Channel level and region level patterns between male mean/std and female mean/std groups.