Decoding Motor Behavior Using Deep Learning and Reservoir Computing

TL;DR

This paper addresses EEG-based motor decoding for non-invasive brain–machine interfaces by combining a CNN front-end for spatial–spectral feature extraction with a fixed Echo State Network reservoir to model temporal dynamics. The ESNNet architecture trains end-to-end the CNN front-end, the ESN input mapping, and the final classifier, achieving strong within-subject performance and competitive cross-subject generalization on skateboard EEG data. Experiments show ESNNet outperforms several CNN baselines in accuracy and offers substantial efficiency advantages, with about 46k parameters and ~0.6 ms per-sample inference on a high-end GPU. The work highlights reservoir computing as a practical, interpretable temporal modeling component for EEG decoding and points to future improvements in reservoir flexibility and cross-subject adaptation.

Abstract

We present a novel approach to EEG decoding for non-invasive brain machine interfaces (BMIs), with a focus on motor-behavior classification. While conventional convolutional architectures such as EEGNet and DeepConvNet are effective in capturing local spatial patterns, they are markedly less suited for modeling long-range temporal dependencies and nonlinear dynamics. To address this limitation, we integrate an Echo State Network (ESN), a prominent paradigm in reservoir computing into the decoding pipeline. ESNs construct a high-dimensional, sparsely connected recurrent reservoir that excels at tracking temporal dynamics, thereby complementing the spatial representational power of CNNs. Evaluated on a skateboard-trick EEG dataset preprocessed via the PREP pipeline and implemented in MNE-Python, our ESNNet achieves 83.2% within-subject and 51.3% LOSO accuracies, surpassing widely used CNN-based baselines. Code is available at https://github.com/Yutiankunkun/Motion-Decoding-Using-Biosignals

Figures (2)

• Figure 1: Illustration of the architecture of the proposed model, which integrates convolutional feature extraction with an ESN for classification. The input time-series signals are first processed by a Temporal Conv2D layer to capture temporal dependencies, followed by a Spatial Conv2D layer that learns spatial correlations across channels. The resulting feature maps are then projected into the ESN block, where the reservoir dynamics enrich the temporal representation. The reservoir $W$ is fixed; the input mapping $W_{\text{in}}$, convolutional layers, and the linear classifier are trained jointly end-to-end, ensuring efficient learning. Finally, the ESN outputs are fed into a classifier, and the prediction is compared with the target labels to complete the supervised learning process.
• Figure 2: Overview of raw EEG data and events.(a) illustration of the temporal structure and variability of neural activity prior to preprocessing. The raw signals highlight characteristic fluctuations, background noise, and inter-channel differences, thereby motivating the subsequent preprocessing procedures described in subsection \ref{['sec:4.1']}. (b) Temporal distribution of labeled events for three skateboarding maneuvers (backside kickturn, frontside kickturn, and pumping) across two conditions with laser and LED. Each dot corresponds to an onset annotation, providing transparency on trial segmentation and ensuring balanced coverage of experimental factors.