MarkovType: A Markov Decision Process Strategy for Non-Invasive Brain-Computer Interfaces Typing Systems

TL;DR

This work addresses fast and accurate noninvasive BCI typing for RSVP by formulating the typing task as a Partially Observable Markov Decision Process (POMDP). MarkovType jointly learns EEG feature extraction and recursive symbol classification across sequences using a simulator, a recurrent core, and a classifier, optimizing a reward $R=\sum_{n=1}^{N} r_n d(n)$ via REINFORCE with a hybrid supervised loss. Compared to Recursive Bayesian discriminative baselines, MarkovType achieves higher accuracy and information transfer rate, while exposing a trade-off between typing speed (number of sequences) and accuracy. The results on a large RSVP benchmark demonstrate the first POMDP formulation for RSVP typing and highlight the potential for improved, real-world noninvasive BCI typing performance.

Abstract

Brain-Computer Interfaces (BCIs) help people with severe speech and motor disabilities communicate and interact with their environment using neural activity. This work focuses on the Rapid Serial Visual Presentation (RSVP) paradigm of BCIs using noninvasive electroencephalography (EEG). The RSVP typing task is a recursive task with multiple sequences, where users see only a subset of symbols in each sequence. Extensive research has been conducted to improve classification in the RSVP typing task, achieving fast classification. However, these methods struggle to achieve high accuracy and do not consider the typing mechanism in the learning procedure. They apply binary target and non-target classification without including recursive training. To improve performance in the classification of symbols while controlling the classification speed, we incorporate the typing setup into training by proposing a Partially Observable Markov Decision Process (POMDP) approach. To the best of our knowledge, this is the first work to formulate the RSVP typing task as a POMDP for recursive classification. Experiments show that the proposed approach, MarkovType, results in a more accurate typing system compared to competitors. Additionally, our experiments demonstrate that while there is a trade-off between accuracy and speed, MarkovType achieves the optimal balance between these factors compared to other methods.

Figures (4)

• Figure 1: MarkovType Model Architecture. It consists of a simulator, a feature extractor $f_{e}$, a core network $f_{h}$ and a classification network $f_{c}$. A single typing trial includes $N$ sequences. Given the target symbol $t$ and the prior over the alphabet $\mathbf{p}_{n\!-\!1}\in\mathbb{R}^{A}$, the simulator extracts a query with $K$ unique symbols $\mathbf{q}_n\in \mathbb{Z}^{K}$ and EEG responses $\mathbf{E}_{n}\in \mathbb{R}^{K\times c\times f}$, where $A$ is the length of the alphabet, $c$ is the number of EEG channels and $f$ is the feature length. $f_e$ extracts features from $\mathbf{E}_{n}$ and maps them over the alphabet using $\mathbf{q}_{n}$ as in Eq. \ref{['eq:feature_mapping']} to obtain $\mathbf{G}_{n} \in \mathbb{R}^{A\times L}$, where $L$ is the feature length. $f_{h}$ forms the hidden states of the current sequence $\mathbf{h}_{n}$ using the alphabet features $\mathbf{G}_{n}$ and the hidden states of the previous sequence ($\mathbf{h}_{n\!-\!1}$), which provides knowledge of the responses to previous queries.
• Figure 2: Distribution of correctly and incorrectly classified symbols over 5 data splits across sequences (1-10). Note that classification sequences terminate when any posterior probability exceeds the threshold or when the maximum number of sequences (in this case, 10) is reached. At each sequence, the blue bar represents correct decisions, and the orange bar represents incorrect decisions. Together, they show the total number of decisions made at each sequence. We observe that although RB methods (e) and (f) make most of their decisions early on, these decisions are mostly incorrect. In contrast, MarkovType methods (a)-(d) tend to make decisions later than those using recursive Bayesian methods but achieve a higher ratio of correct decisions.
• Figure 3: Accuracy across sequences with threshold $\tau$. Lines represent the mean accuracy and confidence bars indicate the standard deviation across the 5 data splits. This plot shows the accuracy of decisions made across sequences. Classification sequences terminate when any posterior probability exceeds the threshold or when the maximum number of sequences is reached. With RB methods, no symbols remain to be classified after the 7th sequence. MarkovType consistently achieves higher accuracy than RB methods.
• Figure 4: Accuracy across sequences without threshold $\tau$, with lines representing the mean accuracy and confidence bars indicating the standard deviation across the 5 data splits. This plot shows the accuracy of symbols with varying numbers of sequences from 1 to 10. Note that when the algorithm includes a threshold, early stopping occurs. Without early stopping, we can observe the upper limit of accuracy with different numbers of sequences. For instance, with a threshold, the model might make an incorrect prediction without an opportunity to correct it in subsequent sequences, resulting in differences in accuracy between cases with and without a threshold. At each sequence, the classification of 1000 symbols in the test set is performed. We see that MarkovType methods perform better at each sequence compared to RB methods, with the MarkovType method using the $\frac{1}{n^2}$ discount factor achieving the highest accuracy.