A Personalized and Adaptable User Interface for a Speech and Cursor Brain-Computer Interface

TL;DR

This work tackles the challenge of enabling independent communication and computer use for people with paralysis through a personalized and adaptable UI for an intracortical BCI. It combines a 22‑month at‑home deployment with iterative, participant‑driven co‑design to produce a multimodal interface that integrates decoded speech, neural cursor control, eye tracking, and gestures within a shared backend. Key contributions include a flexible, cross‑platform architecture, an iterative correction workflow for sentences and words, and design insights that emphasize independence and adaptability over decoding performance alone. The study demonstrates that personalization and multimodal redundancy can sustain daily use and inform future user‑centered BCI assistive technologies, with practical implications for at‑home deployment and platform‑agnostic UI design.

Abstract

Communication and computer interaction are important for autonomy in modern life. Unfortunately, these capabilities can be limited or inaccessible for the millions of people living with paralysis. While implantable brain-computer interfaces (BCIs) show promise for restoring these capabilities, little has been explored on designing BCI user interfaces (UIs) for sustained daily use. Here, we present a personalized UI for an intracortical BCI system that enables users with severe paralysis to communicate and interact with their computers independently. Through a 22-month longitudinal deployment with one participant, we used iterative co-design to develop a system for everyday at-home use and documented how it evolved to meet changing needs. Our findings highlight how personalization and adaptability enabled independence in daily life and provide design implications for developing future BCI assistive technologies.

Figures (6)

• Figure 1: (A) Intracortical brain computer interface system schematic. Microelectrode arrays are surgically placed into the speech motor cortex. Neural information is transmitted through percutaneous wires to a computer system that decodes the neural data into words on a screen. (B) Participant T15 using the BCI system. T15’s personal computer monitors are pictured at the bottom, with the BCI systems monitor positioned on top, displaying the most recent decoded sentence.
• Figure 2: Finite state machine of the BCI interface for participant T15. It depicts different states and how the user navigates between them.
• Figure 3: Speech-related user interface for T15. Subfigures (A)-(F) show a workflow of T15 speaking a sentence and making a correction. A gaze cursor pointer is shown interacting with the buttons on the screen. The following screens are shown: (A) idle (B) speaking (C) sentence rating (D) sentence correction (E) word correction with the selected word highlighted (F) sentence rating with the new corrected sentence. Subfigures (G) and (H) show how T15 can access the sentence history (H) through the menu (G). Subfigures (I) and (J) show accessing speech calibration (J) through the speech menu (I).
• Figure 4: Cursor-related user interface for T15. Subfigures (A)-(C) show how T15 can switch between neural cursor control (B) and gaze control (C) for the BCI interface from the menu (A). Subfigures (D)-(F) show how T15 can use neural cursor (E) or gaze (F) to control his personal computer by selecting options from the menu (D). Subfigures (G) and (H) show accessing neural cursor calibration (H) through the cursor menu (G).
• Figure 5: Participant T15 using the BCI on other platforms. (A) T15 using the BCI on an iPad. (E) T15 using the BCI on his iMac with a floating desktop application. Despite the differing form factor and operating systems, the applications follow the same structure as the original user interface. We show the following screens (B)(F) idle (C)(G) menu (D)(H) sentence correction.