High Performance P300 Spellers Using GPT2 Word Prediction With Cross-Subject Training

TL;DR

The paper addresses slow typing speeds in P300 speller BCIs for ALS by introducing across-subject training and language-model-guided word prediction. It combines GPT-2 with Dijkstra-based word suggestions and smoothing to handle OOV words, using a virtual flashboard and optimized scanning order to boost selection speed. Offline simulations with 78 subjects show large gains in information transfer rate, achieving up to 78.86 bits/min in the best configurations, with GPT2-based methods providing the strongest improvements and near-optimal performance when retrofitted onto traditional flashboards. The work demonstrates the potential of integrating advanced language models into BCIs to reduce calibration needs and significantly improve real-time communication speed, while acknowledging the need for online validation and future upgrades to more capable LLMs.

Abstract

Amyotrophic lateral sclerosis (ALS) severely impairs patients' ability to communicate, often leading to a decline in their quality of life within a few years of diagnosis. The P300 speller brain-computer interface (BCI) offers an alternative communication method by interpreting a subject's EEG response to characters presented on a grid interface. This paper addresses the common speed limitations encountered in training efficient P300-based multi-subject classifiers by introducing innovative "across-subject" classifiers. We leverage a combination of the second-generation Generative Pre-Trained Transformer (GPT2) and Dijkstra's algorithm to optimize stimuli and suggest word completion choices based on typing history. Additionally, we employ a multi-layered smoothing technique to accommodate out-of-vocabulary (OOV) words. Through extensive simulations involving random sampling of EEG data from subjects, we demonstrate significant speed enhancements in typing passages containing rare and OOV words. These optimizations result in approximately 10% improvement in character-level typing speed and up to 40% improvement in multi-word prediction. We demonstrate that augmenting standard row/column highlighting techniques with layered word prediction yields close-to-optimal performance. Furthermore, we explore both "within-subject" and "across-subject" training techniques, showing that speed improvements are consistent across both approaches.

Figures (10)

• Figure 1: Block diagram of the P300 speller with the subject EEG response to character highlights on a flashboard.
• Figure 2: Types of language models used. Starting with the simplest on the left to the most complex on the right.
• Figure 3: a) Probabilistic flashboard highlighting order in a sequential frequency-sorted flashboard. SP denotes word space. b) Sequential flashboard characters shown in Figure \ref{['fig_seq']}a are virtually mapped onto a conventional alphabetical flashboard
• Figure 4: (a) Diagonal flashboard highlighting where most likely characters e, t, a, i ... are organized along diagonals. b) Diagonal highlighted flashboard characters in Figure \ref{['fig_diag']}a are virtually mapped onto a conventional alphabetical flashboard
• Figure 5: a) Huffman scanning steps where 1 and 0 are the direction to move b) Virtual mapping onto a conventional alphabetical flashboard.