ChatBCI: A P300 Speller BCI Leveraging Large Language Models for Improved Sentence Composition in Realistic Scenarios

Abstract

P300 speller BCIs allow users to compose sentences by selecting target keys on a GUI through the detection of P300 component in their EEG signals following visual stimuli. Most P300 speller BCIs require users to spell words letter by letter, or the first few initial letters, resulting in high keystroke demands that increase time, cognitive load, and fatigue. This highlights the need for more efficient, user-friendly methods for faster sentence composition. In this work, we introduce ChatBCI, a P300 speller BCI that leverages the zero-shot learning capabilities of large language models (LLMs) to suggest words from user-spelled initial letters or predict the subsequent word(s), reducing keystrokes and accelerating sentence composition. ChatBCI retrieves word suggestions through remote queries to the GPT-3.5 API. A new GUI, displaying GPT-3.5 word suggestions as extra keys is designed. SWLDA is used for the P300 classification. Seven subjects completed two online spelling tasks: 1) copy-spelling a self-composed sentence using ChatBCI, and 2) improvising a sentence using ChatBCI's word suggestions. Results demonstrate that in Task 1, on average, ChatBCI outperforms letter-by-letter BCI spellers, reducing time and keystrokes by 62.14% and 53.22%, respectively, and increasing information transfer rate by 198.96%. In Task 2, ChatBCI achieves 80.68% keystroke savings and a record 8.53 characters/min for typing speed. Overall, ChatBCI, by employing remote LLM queries, enhances sentence composition in realistic scenarios, significantly outperforming traditional spellers without requiring local model training or storage. ChatBCI's (multi-) word predictions, combined with its new GUI, pave the way for developing next-generation speller BCIs that are efficient and effective for real-time communication, especially for users with communication and motor disabilities.

Figures (5)

• Figure 1: Overview of the proposed ChatBCI. The new keyboard GUI, integrated with remote ChatGPT query, is displayed on the stimulation computer. The recording computer records and processes the acquired EEG signals in real time for P300 detection and key recognition. Key selections detected by the recording computer update the relevant panels in the GUI, displaying the spelled sentence and suggested words.
• Figure 2: The new keyboard GUI developed for ChatBCI. Top-right: the sentence panel (shown within the green box) displays two sentences: the target sentence (e.g., used in the copy-spell task (Task 1)), and the sentence as composed by the user in real-time. Top-left: the experiment panel (shown within the blue box) presents information about the experiment. Bottom: the keyboard panel (shown within the red box) includes character keys, function keys, and $5$ slots on the left and right sides, to display the $10$ candidates suggested by GPT.
• Figure 3: Diagram of the GPT-3.5 query process in ChatBCI. (a): The target sentence and the partially composed text by the user are shown. Spaces are displayed as "-" for visibility. (b): The list of two query messages taking the system and user roles, composed from the partial text. (c): Response from the GPT-3.5-turbo API after sending the messages in (b). (d): List of candidate words formed by splitting and re-formatting the response string in (c). (e): GUI is updated with the candidate words.
• Figure 4: Example of the experimental setup: a subject is using ChatBCI to type a sentence.
• Figure 5: Online session Task 1- Four images representing items relevant to daily life activities are presented to each subject. Subjects are asked to select one image and compose a meaningful sentence relevant to the selected image.