Improving motor imagery decoding methods for an EEG-based mobile brain-computer interface in the context of the 2024 Cybathlon

TL;DR

The paper presents a modular online EEG-based BCI designed for Cybathlon 2024, integrating three motor-imagery tasks to command up to five signals with a novel S4D-based classifier and a real-time, multi-module pipeline. It emphasizes human-centered design, training games, and a mobile feedback interface to enhance co-adaptation and usability. Offline results show strong three-class MI performance, while real-time evaluations reveal competitive but environment-dependent performance, with 73% real-time success after Cybathlon and one task completed during competition. The work highlights modular architecture, low-latency processing, and practical considerations for translating laboratory BCI performance to daily-life use, outlining future improvements such as hybrid feature integration and adaptive calibration for broader applicability.

Abstract

Motivated by the Cybathlon 2024 competition, we developed a modular, online EEG-based brain-computer interface to address these challenges, increasing accessibility for individuals with severe mobility impairments. Our system uses three mental and motor imagery classes to control up to five control signals. The pipeline consists of four modules: data acquisition, preprocessing, classification, and the transfer function to map classification output to control dimensions. We use three diagonalized structured state-space sequence layers as a deep learning classifier. We developed a training game for our pilot where the mental tasks control the game during quick-time events. We implemented a mobile web application for live user feedback. The components were designed with a human-centred approach in collaboration with the tetraplegic user. We achieve up to 84% classification accuracy in offline analysis using an S4D-layer-based model. In a competition setting, our pilot successfully completed one task; we attribute the reduced performance in this context primarily to factors such as stress and the challenging competition environment. Following the Cybathlon, we further validated our pipeline with the original pilot and an additional participant, achieving a success rate of 73% in real-time gameplay. We also compare our model to the EEGEncoder, which is slower in training but has a higher performance. The S4D model outperforms the reference machine learning models. We provide insights into developing a framework for portable BCIs, bridging the gap between the laboratory and daily life. Specifically, our framework integrates modular design, real-time data processing, user-centred feedback, and low-cost hardware to deliver an accessible and adaptable BCI solution, addressing critical gaps in current BCI applications.

Figures (6)

• Figure 1: (a) The pilot wearing the Smarting EEG from mBrainTrain, which collects data and wirelessly sends it to the computer via Bluetooth 2.1. (b) Time course for displaying cues to the pilot to collect data. First, the pilot sees a black screen with a cross marker. Then, a cue marker appears for one second. Depending on the cue, the pilot performs a different task. For example, a left arrow can denote left-hand motor imagery, a right arrow right-hand MI, and a circle rest. After the cue disappears, the pilot begins to perform the imagery or rest task for three seconds. Finally, once the cross marker reappears, they take a three-second break. (c) Event-related synchronisation (ERS) and desynchronization (ERD) over the course of an epoch during left-hand MI at the C4 electrode and right-hand MI at the C3 electrode. (d) Topoplot showing ERD/ERS activity over the entire brain before and during right-hand motor imagery. Activity is pronounced in the left hemisphere around the motor cortex.
• Figure 2: Overview of the brain-computer interface processing pipeline. The blue section illustrates the online and offline pipeline used for data processing, which includes EEG data preprocessing (bandpass filtering between 1-40 Hz, artifact removal via ASR, average re-referencing, and normalization across sessions), as well as feature extraction using Morlet wavelets and common spatial patterns (CSP). Classifier training (orange) is only performed in the offline pipeline. The red sections depict the online pipeline, where EEG signals acquired from the pilot are streamed via LSL, processed through the same preprocessing and feature extraction modules, and classified in real-time. The resulting predictions are transmitted via UDP to the feedback/game module, enabling closed-loop interaction.
• Figure 3: (a) This part of the figure shows one progression of the Dino game. In step 1, the dinosaur is running. When it reaches an object it needs to jump over, such as the cactus, it stops, and a task is indicated via the arrow marker (step 2). When the classifier classifies that the pilot is producing the imagery denoted by the marker, the bar fills up (step 3). If the threshold is reached, the dino jumps over the object (step 4). Finally, the dino keeps running until it reaches the next object (step 5). If the threshold is not reached, then the dino does not jump and loses one health point. (b) Three of the possible sub-games/tasks of the cybathlon game as played in the competition. In the Cursor game, the pilot needs to move the cursor to the highlighted block on the screen and use one of the binary commands to click on it. For the Wheelchair task, the pilot navigates the wheelchair through the room to the exit marked with an arrow. In the Ice Machine task, a robotic arm with a cup is placed under the sensor of an ice machine to collect a set amount of ice. Misclassification leads to the cup tipping and ice falling out. (c) The feedback screen visualises the classification output and thresholds of the transfer function to the pilot. On the left, the large circle shows the different classes being classified via the colourful wedges. If one of them reaches over the threshold in their third, then the corresponding control signal is sent to the game. The two circles on the right fill up based on the class that is classified, and once the circle is full, one of the binary signals is sent to the game.
• Figure 4: (a) On the left is the confusion matrix for the S4D-layer-based model on the training data, and on the right is the confusion matrix for the testing/validation of the data. The 190 cued samples used here correspond to the data obtained during the two Cybathlon competition days in five sessions. (b) Comparison of accuracies for three-class MI. The S4D-layer-based model and EEGEncoder outperform all other baselines. The data is the same as in (a).
• Figure 5: (a) Testing confusion matrix of the offline classifier training for 720 samples from pilot 2. The data was obtained on three consecutive days. The total accuracy is 71%. (b) Online success rates from pilot 2 playing the Dino Game. The pilot achieves 73.17% accuracy, matching the results from offline training performed on the same day.