EEG-Driven AR-Robot System for Zero-Touch Grasping Manipulation

TL;DR

This work tackles the challenge of reliable non-invasive BCI control for assistive robotics by addressing noisy EEG, fixed target mappings, and the absence of real closed-loop validation.It introduces a multimodal closed-loop framework that combines MI-based EEG decoding, smartphone AR neurofeedback, and vision-guided autonomous grasping on a MyCobot 280Pi, enabling hands-free target selection and grasp execution.Key findings show individualized MI calibration achieving $93.1\%$ accuracy and $14.8$ bit/min ITR, AR feedback increasing sustained control to $0.210$ (SCI) and elevating ITR to $21.3$ bits/min, and a $97.2\%$ closed-loop grasping success rate, demonstrating robust end-to-end performance.The approach holds practical significance for assistive robotics by delivering ecological AR interaction, seamless BCI–AR–Robot integration, and zero-touch operation in real-world tasks.

Abstract

Reliable brain-computer interface (BCI) control of robots provides an intuitive and accessible means of human-robot interaction, particularly valuable for individuals with motor impairments. However, existing BCI-Robot systems face major limitations: electroencephalography (EEG) signals are noisy and unstable, target selection is often predefined and inflexible, and most studies remain restricted to simulation without closed-loop validation. These issues hinder real-world deployment in assistive scenarios. To address them, we propose a closed-loop BCI-AR-Robot system that integrates motor imagery (MI)-based EEG decoding, augmented reality (AR) neurofeedback, and robotic grasping for zero-touch operation. A 14-channel EEG headset enabled individualized MI calibration, a smartphone-based AR interface supported multi-target navigation with direction-congruent feedback to enhance stability, and the robotic arm combined decision outputs with vision-based pose estimation for autonomous grasping. Experiments are conducted to validate the framework: MI training achieved 93.1 percent accuracy with an average information transfer rate (ITR) of 14.8 bit/min; AR neurofeedback significantly improved sustained control (SCI = 0.210) and achieved the highest ITR (21.3 bit/min) compared with static, sham, and no-AR baselines; and closed-loop grasping achieved a 97.2 percent success rate with good efficiency and strong user-reported control. These results show that AR feedback substantially stabilizes EEG-based control and that the proposed framework enables robust zero-touch grasping, advancing assistive robotic applications and future modes of human-robot interaction.

Figures (7)

• Figure 1: System overview of the proposed BCI–AR–Robot closed-loop grasping framework, integrating (i) EEG-based motor imagery decoding, (ii) AR-based interaction and feedback, and (iii) vision-guided robotic execution.
• Figure 2: EmotivBCI training and validation interfaces: (a) Training preparation; (b) During training-End of training (scored)-Saving of training results.
• Figure 3: MI training timeline.
• Figure 4: Illustration of Experiment 2. Four BCI-AR feedback conditions and the trial timeline.
• Figure 5: BCI-AR experimental setup for closed-loop robotic grasping.