Enhancing Interpretability of AR-SSVEP-Based Motor Intention Recognition via CNN-BiLSTM and SHAP Analysis on EEG Data

TL;DR

This work tackles the limited practicality and interpretability of traditional SSVEP-based BCIs by integrating augmented reality stimuli via HoloLens2 with a MACNN-BiLSTM model enhanced by multi-head attention. The system achieves high real-time motor-intention recognition accuracy (up to 94.67% at 1.5 s) and leverages SHAP to reveal key EEG features driving decisions, notably occipital Alpha/Beta power metrics. The combination of AR visualization, deep learning, and explainable AI advances BCI usability for neurorehabilitation and clinical deployment, while outlining directions for clinical validation and cross-subject generalization.

Abstract

Patients with motor dysfunction show low subjective engagement in rehabilitation training. Traditional SSVEP-based brain-computer interface (BCI) systems rely heavily on external visual stimulus equipment, limiting their practicality in real-world settings. This study proposes an augmented reality steady-state visually evoked potential (AR-SSVEP) system to address the lack of patient initiative and the high workload on therapists. Firstly, we design four HoloLens 2-based EEG classes and collect EEG data from seven healthy subjects for analysis. Secondly, we build upon the conventional CNN-BiLSTM architecture by integrating a multi-head attention mechanism (MACNN-BiLSTM). We extract ten temporal-spectral EEG features and feed them into a CNN to learn high-level representations. Then, we use BiLSTM to model sequential dependencies and apply a multi-head attention mechanism to highlight motor-intention-related patterns. Finally, the SHAP (SHapley Additive exPlanations) method is applied to visualize EEG feature contributions to the neural network's decision-making process, enhancing the model's interpretability. These findings enhance real-time motor intention recognition and support recovery in patients with motor impairments.

Figures (7)

• Figure 1: AR-BCI motor intention recognition system.
• Figure 2: Visual stimulation interface.
• Figure 3: Process of SSVEP stimulus test.
• Figure 4: Structure of MACNN-BiLSTM.
• Figure 5: Average accuracy across subjects (0.5--1.5 s).