CSP-Net: Common Spatial Pattern Empowered Neural Networks for EEG-Based Motor Imagery Classification

TL;DR

Two CSP-empowered neural networks (CSP-Nets), which integrate knowledge-driven CSP filters with data-driven CNNs to enhance the performance in MI classification and demonstrate the advantage of integrating knowledge-driven traditional machine learning with data-driven deep learning in EEG-based brain-computer interfaces.

Abstract

Electroencephalogram-based motor imagery (MI) classification is an important paradigm of non-invasive brain-computer interfaces. Common spatial pattern (CSP), which exploits different energy distributions on the scalp while performing different MI tasks, is very popular in MI classification. Convolutional neural networks (CNNs) have also achieved great success, due to their powerful learning capabilities. This paper proposes two CSP-empowered neural networks (CSP-Nets), which integrate knowledge-driven CSP filters with data-driven CNNs to enhance the performance in MI classification. CSP-Net-1 directly adds a CSP layer before a CNN to improve the input discriminability. CSP-Net-2 replaces a convolutional layer in CNN with a CSP layer. The CSP layer parameters in both CSP-Nets are initialized with CSP filters designed from the training data. During training, they can either be kept fixed or optimized using gradient descent. Experiments on four public MI datasets demonstrated that the two CSP-Nets consistently improved over their CNN backbones, in both within-subject and cross-subject classifications. They are particularly useful when the number of training samples is very small. Our work demonstrates the advantage of integrating knowledge-driven traditional machine learning with data-driven deep learning in EEG-based brain-computer interfaces.

Figures (9)

• Figure 1: t-SNE visualization of (a) the raw EEG trials; and, (b) the CSP-filtered trials. Different shapes (colors) represent different classes.
• Figure 2: Our proposed CSP-Nets. (a) Traditional CSP filters are used to initialize the CSP layer in CSP-Nets. (b) CSP-Net-1, which directly adds a CSP layer before a CNN backbone. (c) CSP-Net-2, illustrated using EEGNet EEGNet (Table 1 in Supplementary Materials); the DepthwiseConv2D layer in its depthwise spatial filter block is replaced by a CSP layer.
• Figure 3: Accuracy improvements of CSP-Nets at different training data ratios on MI4C. (a) within-subject classification; and, (b) cross-subject classification.
• Figure 4: Accuracy improvements of CSP-Nets at different training data ratios on MI2C. (a) within-subject classification; and, (b) cross-subject classification.
• Figure 5: Accuracy improvements of CSP-Nets at different training data ratios on MI14S. (a) within-subject classification; and, (b) cross-subject classification.