EEG-Deformer: A Dense Convolutional Transformer for Brain-computer Interfaces

TL;DR

Visualization results show that EEG-Deformer learns from neurophysiologically meaningful brain regions for the corresponding cognitive tasks, demonstrating that it either outperforms or performs comparably to existing state-of-the-art methods.

Abstract

Effectively learning the temporal dynamics in electroencephalogram (EEG) signals is challenging yet essential for decoding brain activities using brain-computer interfaces (BCIs). Although Transformers are popular for their long-term sequential learning ability in the BCI field, most methods combining Transformers with convolutional neural networks (CNNs) fail to capture the coarse-to-fine temporal dynamics of EEG signals. To overcome this limitation, we introduce EEG-Deformer, which incorporates two main novel components into a CNN-Transformer: (1) a Hierarchical Coarse-to-Fine Transformer (HCT) block that integrates a Fine-grained Temporal Learning (FTL) branch into Transformers, effectively discerning coarse-to-fine temporal patterns; and (2) a Dense Information Purification (DIP) module, which utilizes multi-level, purified temporal information to enhance decoding accuracy. Comprehensive experiments on three representative cognitive tasks-cognitive attention, driving fatigue, and mental workload detection-consistently confirm the generalizability of our proposed EEG-Deformer, demonstrating that it either outperforms or performs comparably to existing state-of-the-art methods. Visualization results show that EEG-Deformer learns from neurophysiologically meaningful brain regions for the corresponding cognitive tasks. The source code can be found at https://github.com/yi-ding-cs/EEG-Deformer.

Figures (9)

• Figure 1: Comparison of network architectures between ViT dosovitskiy2021an, EEG-Conformer 9991178, and our proposed EEG-Deformer. We propose a novel Hierarchical Coarse-to-Fine Transformer (HCT). Additionally, we have designed Information Purification Unit (IP-Unit, denoted by IP in the figure) for each HCT layer with dense connections to further boost EEG decoding performance.
• Figure 2: The network structure of EEG-Deformer. EEG-Deformer consists of three main parts: (1) Shallow feature encoder, (2) Hierarchical coarse-to-fine-Transformer (HCT), and (3) Dense information purification (DIP). The fine-grained representations from each HCT will be passed to Information Purification Unit (IP-Unit) and concatenated (C) to the final embedding.
• Figure 3: The structure of the shallow feature encoder. After standard CNN layers, the representation is rearranged into kernel by feature, and a position encoding is added onto it.
• Figure 4: The structure of the hierarchical coarse-to-fine Transformer: The right side is capable of learning coarse-grained temporal information through self-attention, while the left side is the newly added FTL.
• Figure 5: Saliency maps of attention classification tasks. (a)-(e) are five representative subjects and (f) is the average of all the subjects. The most informative areas are primarily located in the frontal (Fp1, F1, and AFF6) and parietal (CP5, P7, and P4) regions. The location of each EEG electrode can be found according to its name on the saliency map.