Motor Imagery Teleoperation of a Mobile Robot Using a Low-Cost Brain-Computer Interface for Multi-Day Validation

TL;DR

This work tackles the practical deployment of motor imagery-based BCIs for real-time teleoperation of a mobile robot using a low-cost, 16-channel EEG system. It introduces a fine-tuned deep neural network built on a sliding-window approach (ATCNet-derived) to decode four MI commands with minimal daily data, adapting to day-to-day brain signal variability. Across multi-day validation with four participants, the method achieves about 75% validation accuracy and 62% real-time robot-control accuracy, while reducing calibration data by roughly 70% per day. The results demonstrate that MI-BCI robotic control can be made more accessible and robust for real-world use, reducing fatigue and hardware costs without sacrificing essential performance.

Abstract

Brain-computer interfaces (BCI) have the potential to provide transformative control in prosthetics, assistive technologies (wheelchairs), robotics, and human-computer interfaces. While Motor Imagery (MI) offers an intuitive approach to BCI control, its practical implementation is often limited by the requirement for expensive devices, extensive training data, and complex algorithms, leading to user fatigue and reduced accessibility. In this paper, we demonstrate that effective MI-BCI control of a mobile robot in real-world settings can be achieved using a fine-tuned Deep Neural Network (DNN) with a sliding window, eliminating the need for complex feature extractions for real-time robot control. The fine-tuning process optimizes the convolutional and attention layers of the DNN to adapt to each user's daily MI data streams, reducing training data by 70% and minimizing user fatigue from extended data collection. Using a low-cost (~$3k), 16-channel, non-invasive, open-source electroencephalogram (EEG) device, four users teleoperated a quadruped robot over three days. The system achieved 78% accuracy on a single-day validation dataset and maintained a 75% validation accuracy over three days without extensive retraining from day-to-day. For real-world robot command classification, we achieved an average of 62% accuracy. By providing empirical evidence that MI-BCI systems can maintain performance over multiple days with reduced training data to DNN and a low-cost EEG device, our work enhances the practicality and accessibility of BCI technology. This advancement makes BCI applications more feasible for real-world scenarios, particularly in controlling robotic systems.

Figures (6)

• Figure 1: The proposed approach for teleoperating the mobile robot via the BCI. (A) The robot during a run being teleoperated via the BCI device. (B) An overview of the proposed methodology utilizing a 16-channel BCI via MI with DNN and fine-tuning. Users A and B, and Users C and D were paired to create pre-trained models. Each user's fine-tuned model was developed by further training the pre-trained model that included their data. (C) The visualization of the real-time EEG decoding output where the real robot can be identified in (A) with corresponding colored diamonds matching each segment. A video of this research can be accessed: https://youtu.be/1rR7YFRAJhs?si=6Yj7lB1G7XL-0VAh.
• Figure 2: (A) EEG node positions used during the investigation for the 16 channel device. (B) A view of the BCI, GUI and user during the investigation. (C) Point of View from the camera mounted on quadruped robot displayed to the user from the GUI. The crosshair size has been increased slightly compared to the real implementation for improved visualization.
• Figure 3: (A) An overview of the hardware setup for the MI-BCI controller. (B) Investigation procedure highlighting each phase of data collection and validation which lead to real robot teleoperation via EEG MI-BCI. (C) An overview of the blocks used to create the DNN where the orange blocks represent the fine-tuned layers and others are frozen. Sliding Window (I) includes the input from ATCNet. Sliding Window (II) is the layer from the original ATCNet model.
• Figure 4: Accuracy of real-time robot control on Day 0 after long-term data collection, shown as a function of the number of runs. Note that in these robot control datasets, User A-C's concentration decreased compared to the validation datasets due to fatigue, which was mitigated by our fine-tuned approach.
• Figure 5: Average accuracy of real-time robot control after short-term data collection on new days, plotted as a function of the number of runs. High accuracy was maintained during robot control due to the DNN optimized for each user and each day.