Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Spatial-Spectral and Temporal Dual Prototype Network for Motor Imagery Brain-Computer Interface

Can Han, Chen Liu, Yaqi Wang, Crystal Cai, Jun Wang, Dahong Qian

TL;DR

This work designs a lightweight attention mechanism to uniformly model the spatial-spectral relationships across multiple EEG electrodes, enabling the extraction of powerful spatial-spectral features and introduces dual prototype learning to optimize the feature space distribution and training process, thereby improving the model's generalization ability on small-sample MI datasets.

Abstract

Motor imagery electroencephalogram (MI-EEG) decoding plays a crucial role in developing motor imagery brain-computer interfaces (MI-BCIs). However, decoding intentions from MI remains challenging due to the inherent complexity of EEG signals relative to the small-sample size. To address this issue, we propose a spatial-spectral and temporal dual prototype network (SST-DPN). First, we design a lightweight attention mechanism to uniformly model the spatial-spectral relationships across multiple EEG electrodes, enabling the extraction of powerful spatial-spectral features. Then, we develop a multi-scale variance pooling module tailored for EEG signals to capture long-term temporal features. This module is parameter-free and computationally efficient, offering clear advantages over the widely used transformer models. Furthermore, we introduce dual prototype learning to optimize the feature space distribution and training process, thereby improving the model's generalization ability on small-sample MI datasets. Our experimental results show that the SST-DPN outperforms state-of-the-art models with superior classification accuracy (84.11% for dataset BCI4-2A, 86.65% for dataset BCI4-2B). Additionally, we use the BCI3-4A dataset with fewer training data to further validate the generalization ability of the proposed SST-DPN. We also achieve superior performance with 82.03% classification accuracy. Benefiting from the lightweight parameters and superior decoding accuracy, our SST-DPN shows great potential for practical MI-BCI applications. The code is publicly available at https://github.com/hancan16/SST-DPN.

A Spatial-Spectral and Temporal Dual Prototype Network for Motor Imagery Brain-Computer Interface

TL;DR

This work designs a lightweight attention mechanism to uniformly model the spatial-spectral relationships across multiple EEG electrodes, enabling the extraction of powerful spatial-spectral features and introduces dual prototype learning to optimize the feature space distribution and training process, thereby improving the model's generalization ability on small-sample MI datasets.

Abstract

Motor imagery electroencephalogram (MI-EEG) decoding plays a crucial role in developing motor imagery brain-computer interfaces (MI-BCIs). However, decoding intentions from MI remains challenging due to the inherent complexity of EEG signals relative to the small-sample size. To address this issue, we propose a spatial-spectral and temporal dual prototype network (SST-DPN). First, we design a lightweight attention mechanism to uniformly model the spatial-spectral relationships across multiple EEG electrodes, enabling the extraction of powerful spatial-spectral features. Then, we develop a multi-scale variance pooling module tailored for EEG signals to capture long-term temporal features. This module is parameter-free and computationally efficient, offering clear advantages over the widely used transformer models. Furthermore, we introduce dual prototype learning to optimize the feature space distribution and training process, thereby improving the model's generalization ability on small-sample MI datasets. Our experimental results show that the SST-DPN outperforms state-of-the-art models with superior classification accuracy (84.11% for dataset BCI4-2A, 86.65% for dataset BCI4-2B). Additionally, we use the BCI3-4A dataset with fewer training data to further validate the generalization ability of the proposed SST-DPN. We also achieve superior performance with 82.03% classification accuracy. Benefiting from the lightweight parameters and superior decoding accuracy, our SST-DPN shows great potential for practical MI-BCI applications. The code is publicly available at https://github.com/hancan16/SST-DPN.
Paper Structure (27 sections, 13 equations, 12 figures, 7 tables)

This paper contains 27 sections, 13 equations, 12 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Deep Learning based EEG-MI Decoding
  4. Prototype Learning
  5. Method
  6. EEG Representation
  7. Adaptive Spatial-Spectral Fusion
  8. LightConv for Spatial-Spectral Representation
  9. Spatial-Spectral Attention
  10. Multi-scale Variance Pooling
  11. Dual Prototype Learning
  12. Experiments and results
  13. Evaluation Datasets
  14. Experimental Setups
  15. Experimental Details
  16. ...and 12 more sections

Figures (12)

  • Figure 1: The overall framework of the proposed SST-DPN. Our SST-DPN takes raw EEG signals as input, extracts features through the ASSF and MVP modules, and makes classification decisions via the DPL module, enabling its application in human-computer interaction control.
  • Figure 2: The detailed diagram of the implementation of spatial-spectral attention.
  • Figure 3: An illustration of the proposed VarPool layer.
  • Figure 4: An illustration of the feature space distribution for CE loss, PL, and our proposed DPL. CE loss only ensures that features of different classes are generally separable, resulting in a loose feature space. PL increases the compactness of features within the same class. Building on PL, our DPL further enlarges the margins between features of different classes.
  • Figure 5: Accuracy ablation results for each subject on Dataset I. "Avg" indicates average accuracy.
  • ...and 7 more figures