NOIR 2.0: Neural Signal Operated Intelligent Robots for Everyday Activities

TL;DR

NOIR 2.0 tackles decodability and usability challenges in EEG-based brain-robot interfaces by introducing a modular decoding pipeline (What/How/Where) and a robot with parameterized primitive skills. It couples SSVEP-based object selection and MI-based skill/parameter control with EMG-based confirmation, and integrates retrieval-based few-shot planning via GPT-4o alongside one-shot parameter learning with DINOv2. The approach yields substantial reductions in total task time and human decision time, and improves decoding accuracy (e.g., object selection ~0.88 task-time, skill selection ~0.61 task-time), while enabling users to skip routine decisions in a majority of trials. The work highlights practical impact for assistive robotics by reducing training requirements and cognitive load, and discusses ethical/safety considerations for deployment in real environments.

Abstract

Neural Signal Operated Intelligent Robots (NOIR) system is a versatile brain-robot interface that allows humans to control robots for daily tasks using their brain signals. This interface utilizes electroencephalography (EEG) to translate human intentions regarding specific objects and desired actions directly into commands that robots can execute. We present NOIR 2.0, an enhanced version of NOIR. NOIR 2.0 includes faster and more accurate brain decoding algorithms, which reduce task completion time by 46%. NOIR 2.0 uses few-shot robot learning algorithms to adapt to individual users and predict their intentions. The new learning algorithms leverage foundation models for more sample-efficient learning and adaptation (15 demos vs. a single demo), significantly reducing overall human time by 65%.

Figures (4)

• Figure 1: NOIR 2.0 system overview. Following NOIR, NOIR 2.0 implements a modular pipeline for decoding goals from human brain signals, and a robotic system with a library of primitive skills. While minimizing the effort needed for decoding, the robot system is capable of learning to anticipate the goals that humans intend to achieve.
• Figure 2: A structured framework for interpreting human objectives from EEG signals that include: (a) What object to manipulate, determined by SSVEP signals using CCA classifiers; (b) How to engage with the object; and (c) Where to interact with the object, deciphered through MI signals using FBCSP+SVM algorithms. A safety mechanism also monitors muscle tension from the jaw to either confirm or reject the decoding results.
• Figure 3: Example cursor trajectories during parameter selection on a graphical user interface: (a) shows how the mouse cursor moves using binary control in NOIR. (b) shows how the mouse cursor moves in all four directions through continuous, closed-loop control in NOIR 2.0.
• Figure 4: Overview of the robot learning method for NOIR 2.0.