Control of a commercially available vehicle by a tetraplegic human using a brain-computer interface

Abstract

Brain-computer interfaces (BCIs) read neural signals directly from the brain to infer motor planning and execution. However, the implementation of this technology has been largely limited to laboratory settings, with few real-world applications. We developed a BCI system to drive a vehicle in both simulated and real-world environments. We demonstrate that an individual with tetraplegia, implanted with intracortical BCI electrodes in the posterior parietal cortex (PPC) and the hand knob region of the motor cortex (MC), reacts at least as fast and precisely as motor intact participants. This BCI participant, living in California, could also remotely drive a Ford Mustang Mach-E vehicle in Michigan. Our teledriving tasks relied on cursor movement control for speed and steering in a closed urban test facility and through a predefined obstacle course. These two tasks serve as a proof-of-concept that takes into account the safety and feasibility of BCI-controlled driving. The final BCI system added click control for full-stop braking and thus enabled bimanual cursor-and-click control for simulated town driving with the same proficiency level as the motor intact control group through a virtual town with traffic. This first-of-its-kind implantable BCI application not only highlights the versatility and innovative potentials of BCIs but also illuminates the promising future for the development of life-changing solutions to improve independent mobility for those who suffer catastrophic neurological injury.

Figures (7)

• Figure 1: The BCI control system diagram for teledriving a Ford Mustang Mach-E vehicle. Our setup included the decoder and display computers at the BCI test site in California and the in-vehicle controller computer at the vehicle test site in Michigan. Brain signals were recorded from BCI-JJ's posterior parietal cortex (PPC) and the hand knob region of the motor cortex (MC) via the intracortical NeuroPort Electrodes © Blackrock Neurotech and processed by the NeuroPort system © Blackrock Neurotech. The decoder computer extracted the neural features from the acquired signals, decoded the motor intention, and generated a corresponding motor command. This motor command was transmitted to the display computer and then sent to the in-vehicle controller via TCP and UDP. At the vehicle test site, the in-vehicle controller computer executed the motor command for the vehicle. This computer also recorded a live-stream video from a camera mounted on the vehicle. The video feedback and the vehicle state were transferred to the display computer at the BCI test site for BCI-JJ to watch and respond in real-time.
• Figure 2: Detailed flowcharts of the BCI-enabled driving system. (A) The detailed mechanism of BCI neural signal decoding and driving control of either a Ford Mustang Mach-E vehicle remotely or a CARLA simulated vehicle. (B) The vehicle control architecture of a Ford Mach-E vehicle.
• Figure 3: Display of five tasks and the driving routes. (A) Simple reaction time task. (B) Simulated braking reaction time task in CARLA 0.9.13. (C) BCI-controlled teledriving of a Mach-E without traffic in the Mcity test facility. (D) Four teledriving routes in Mcity. (E) BCI-controlled teledriving of a Mach-E without traffic on an obstacle course. (F) Four teledriving route options on the loop-shaped obstacle course. (G) Simulated town driving with traffic in the CARLA Leaderboard 2.0. (H) One simulated driving route in Town 12 of CARLA.
• Figure 4: BCI trial-based simple and braking reaction time task results among different hand effectors of BCI-JJ. (A and B) For the simple reaction time task, we collected 10 runs for each of BCI-JJ's six effectors (i.e., right and left index fingers, ring fingers, and power grips) consisting of 40 GO trials and 10 randomly interleaved NO-GO catch trials per run. We compared simple reaction times within the valid range from 50 ms to 1000 ms in GO trials/phases among the six effectors. We also conducted comparisons of simple reaction performance measures (i.e., accuracy, sensitivity, specificity) among these effectors. (C and D) For the braking reaction time task, we collected 10 runs per right or left index finger consisting of 40 trials per run, with one NO-GO phase and one GO phase in each trial. We compared valid braking reaction times and reaction performance measures between these two effectors. Box charts whose shaded notches around the median lines do not overlap have different medians at the corrected $5\%$ significance level.
• Figure 5: Trial-based simple and braking reaction time task results between BCI-JJ and the motor intact control group with the same right index finger effector. We recruited 20 motor intact participants with no gender bias and an average age of $50 \pm 15$, the same as the age of BCI-JJ at the time of testing. For the simple reaction time task, we collected 10 runs from BCI-JJ and 4 runs from each of the 20 motor intact participants, with each run consisting of 40 GO trials and 10 randomly interleaved NO-GO catch trials. For the braking reaction time task, we collected 10 runs from BCI-JJ and 5 runs from each motor intact participant, consisting of 40 trials per run, with one NO-GO phase and one GO phase in each trial. We compared the valid reaction times between 50 ms and 1000 ms in GO trials/phases between BCI-JJ and the motor intact control group for (A) the simple reaction time task and (B) the braking reaction time task. We also conducted group comparisons of reaction accuracy, sensitivity and specificity for (C) the simple reaction time task and (D) the braking reaction time task. Box charts whose shaded notches around the median lines do not overlap have different medians at the corrected $5\%$ significance level.