Decoding Imagined Handwriting from EEG

TL;DR

This work interrogates the viability of decoding imagined handwriting from non-invasive EEG, addressing limitations in prior studies that relied on known onset or actual motion. Using a 4-letter set and a motor-imagery handwriting paradigm, the authors show that motor imagery yields above-chance decoding, but performance is strongly limited by unknown onset timing and low single-trial SNR. A key finding is that scaling training data provides diminishing returns, while averaging evaluation trials to boost SNR dramatically improves accuracy, revealing a noise ceiling in single-trial EEG. The study highlights the practical challenges for EEG-based handwriting BCIs and points to onset detection and SNR enhancement as essential future directions for real-world use.

Abstract

Patients with extreme forms of paralysis face challenges in communication, adversely impacting their quality of life. Recent studies have reported higher-than-chance performance in decoding handwritten letters from EEG signals, potentially allowing these subjects to communicate. However, all prior works have attempted to decode handwriting from EEG during actual motion. Furthermore, they assume that precise movement-onset is known. In this work, we focus on settings closer to real-world use where either movement onset is not known or movement does not occur at all, fully utilizing motor imagery. We show that several existing studies are affected by confounds that make them inapplicable to the imagined handwriting setting. We also investigate how sample complexity affects handwriting decoding performance, guiding future data collection efforts. Our work shows that (a) Sample complexity analysis in single-trial EEG reveals a noise ceiling, which can be alleviated by averaging over trials. (b) Knowledge of movement-onset is crucial to reported performance in prior works. (c) Fully imagined handwriting can be decoded from EEG with higher-than-chance performance. Taken together, these results highlight both the unique challenges and avenues to pursue to build a practical EEG-based handwriting BCI.

Figures (7)

• Figure 1: One of the 10 ICs used for prediction in prior work. The IC clearly represents the horizontal motion of the eyes tracking the letters in different boxes. This IC alone achieves 85.6% accuracy.
• Figure 2: Experimental design for a single trial. The participant fixates on the monitor in front while writing on the tablet on the desk. One of the four letters is shown for 800 ms, followed by a blank screen for a randomly chosen period between 400-600 ms. Then a fixation cross appears on the screen for 1000 ms, during which the participant writes the letter on the tablet while looking at the cross. This is followed by a blank screen for 500 ms, after which the next trial begins.
• Figure 3: EEGNet architecture
• Figure 4: Subject-wise decoding performance on 3 different settings. Performance drops significantly when motor activity onset is unknown, even with actual motion; indicating a key issue to overcome for decoding imagined handwriting from EEG
• Figure 5: With knowledge of true movement onset (movement-centered), decoding performance exhibits diminishing gains as we scale up training data. Performance in the fixation-centered setting appears to follow this trend, albeit at a later stage.