Low-Latency Neural Inference on an Edge Device for Real-Time Handwriting Recognition from EEG Signals

TL;DR

This work demonstrates that advanced machine learning combined with informative EEG feature extraction can overcome barriers, enabling real-time, high-accuracy neural decoding on portable edge devices, establishing a pathway for accurate, low-latency, and fully portable non-invasive BCIs supporting real-time communication.

Abstract

Brain-computer interfaces (BCIs) offer a pathway to restore communication for individuals with severe motor or speech impairments. Imagined handwriting provides an intuitive paradigm for character-level neural decoding, bridging the gap between human intention and digital communication. While invasive approaches such as electrocorticography (ECoG) achieve high accuracy, their surgical risks limit widespread adoption. Non-invasive electroencephalography (EEG) offers safer and more scalable alternatives but suffers from low signal-to-noise ratio and spatial resolution, constraining its decoding precision. This work demonstrates that advanced machine learning combined with informative EEG feature extraction can overcome these barriers, enabling real-time, high-accuracy neural decoding on portable edge devices. A 32-channel EEG dataset was collected from fifteen participants performing imagined handwriting. Signals were preprocessed with bandpass filtering and artifact subspace reconstruction, followed by extraction of 85 time-, frequency-, and graphical-domain features. A hybrid architecture, EEdGeNet, integrates a Temporal Convolutional Network with a multilayer perceptron trained on the extracted features. When deployed on an NVIDIA Jetson TX2, the system achieved 89.83 percent accuracy with 914.18 ms per-character latency. Selecting only ten key features reduced latency by 4.5 times to 202.6 ms with less than 1 percent loss in accuracy. These results establish a pathway for accurate, low-latency, and fully portable non-invasive BCIs supporting real-time communication.

Figures (6)

• Figure 1: Proposed framework for real-time imagined handwriting decoding from non-invasive EEG signals. (a) Illustration of a motor-impaired person wearing a multi-channel EEG headcap for real-time neural decoding. (b) Comparative analysis of prior studies based on: non-invasive signal acquisition, imagined handwriting/speech decoding, real-time inference compatibility, edge-level deployment feasibility, and classification performance. The proposed approach is the only method that meets all criteria concurrently. (c) Accuracy comparison with state-of-the-art EEG models on 85 features extracted from imagined handwriting data, showing the superior performance of our proposed model (EEdGeNet) for edge-based inference.
• Figure 2: Proposed methodology for low-latency neural inference on an edge device for real-time handwriting recognition from EEG signals. (a) The entire pipeline for training (yellow) and inference (green), including preprocessing, feature extraction, and per-character prediction. (b) Data collection setup showing a participant wearing the Emotiv Flex 2.0 EEG headcap during data acquisition. (c) Electrode layout based on the international 10–20 system, with utilized channels highlighted in green. (d) Real-time inference on the NVIDIA Jetson TX2 with a batch size of 1.
• Figure 3: (a) Raw EEG signals. (b) Preprocessed EEG signals. The 32-channel EEG data from a participant’s sample for the character 'v' is shown before and after preprocessing. Bandpass filtering (1–50 Hz) and Artifact Subspace Reconstruction (ASR) were applied to effectively remove noise and artifacts, improving overall signal quality.
• Figure 4: Proposed EEdGeNet model architecture integrating a Temporal Convolutional Block (TCB) and a Dense Transformation Block (DTB). The TCB (highlighted in yellow) captures temporal dependencies via dilated convolutions, while the DTB (highlighted in blue) performs hierarchical non-linear feature transformations. Abbreviations: BN—Batch Normalization, DR—Dropout layer, EL—ELU activation, and $D$—dilation rate.
• Figure 5: Impact of feature selection using Pearson correlation coefficients on classification accuracy, inference latency, and energy consumption on the NVIDIA Jetson TX2. A compact feature set comprising Hjorth mobility, root mean square, Hjorth complexity, delta power, graphical_A ellipse area, mean dominant frequency, graphical_D3 ellipse area, graphical_D1 ellipse area, graphical_D2 ellipse area, and graphical_D4 ellipse area results in (a) a $\sim$1% reduction in accuracy relative to the full 85-feature set evaluated on the proposed EEdGeNet model, (b) a 4.51$\times$ improvement in per-character inference time, and (c) a 5.14$\times$ reduction in per-character inference energy. This trade-off highlights an efficient balance between computational cost and predictive performance, well-suited for edge deployment on the Jetson TX2.