Deep learning with convolutional neural networks for EEG decoding and visualization

TL;DR

The study evaluates end-to-end ConvNets for EEG decoding against a well-established FBCSP baseline, using multiple architectures (Deep, Shallow, Hybrid, Residual) and training strategies (trial-wise and cropped) on motor-decoding tasks from two EEG datasets. It shows ConvNets can match or slightly exceed FBCSP performance when modern deep-learning practices (batch normalization, dropout, ELUs) are applied and cropped training is used, while also enabling novel visualization methods to map learned band-power features to brain regions. The work provides practical guidance on architecture and training choices for EEG decoding, demonstrates the value of end-to-end learning for brain signals, and introduces visualization techniques that advance interpretable brain-signal decoding and mapping. These findings pave the way for broader adoption of ConvNets in EEG-based BCI and neuroscience research, with potential benefits for online decoding, transfer learning, and cross-modal brain signal analysis.

Abstract

PLEASE READ AND CITE THE REVISED VERSION at Human Brain Mapping: http://onlinelibrary.wiley.com/doi/10.1002/hbm.23730/full Code available here: https://github.com/robintibor/braindecode

Figures (19)

• Figure 1: Deep ConvNet architecture. EEG input (at the top) is progressively transformed towards the bottom, until the final classifier output. Black cuboids: inputs/feature maps; brown cuboids: convolution/pooling kernels. The corresponding sizes are indicated in black and brown, respectively. Note that in this schematics, proportions of maps and kernels are only approximate.
• Figure 2: Shallow ConvNet architecture. Conventions as in Figure \ref{['fig:deep-net']}.
• Figure 3: Residual Block. Residual block used in the ResNet architecture and as described in original paper he_deep_2015, see Figure 2) with identity shortcut option A, except using ELU instead of ReLU nonlinearities. See Section \ref{['subsec:resnet']} for explanation.
• Figure 4: Multiple-crop prediction used for cropped training. In this toy example, a trial with the sample values 1,2,3,4,5,6,7 is cut into three crops of length 5 and these crops are passed through a convolutional network with two convolutional layers and one dense layer. The convolutional layers both have kernel size 2, the second one additionally uses a stride of 2. Filters for both layers and the final dense layer have values 1,1. Red indicates intermediate outputs that were computed multiple times in the naïve implementation. Note that both implementations result in the same final outputs.
• Figure 5: ConvNet Receptive Fields Schema. Showing the outputs, inputs and receptive fields of one unit per layer. Colors indicate different units. Filled rectangles are individual units, solid lines indicate their direct input from the layer before. Dashed lines indicate the corresponding receptive field in all previous layers including the original input layer. The receptive field of a unit contains all inputs that are used to compute the unit's output. The receptive fields get larger with increasing depth of the layer. Note that this is only a schema and exact dimensions are not meaningful in this figure.