Project Jenkins: Turning Monkey Neural Data into Robotic Arm Movement, and Back

TL;DR

The paper addresses bidirectional brain–computer interfacing by decoding macaque motor cortex activity into planar arm velocities and encoding intended movements into synthetic neural data. It combines an MLP-based decoder (using a $50$-bin neural history) with a transformer-based encoder (with a $40$-bin look-ahead) and demonstrates closed-loop operation using Koch v1.1 arms. Key contributions include achieving $R^2 \approx 0.9$ for decoding, stable spike-generation via transformer encoding over long horizons, and an open-source toolkit plus an in-browser interface for real-time synthetic data generation. The work advances flexible, generalizable BCIs and provides reproducible resources to spur progress in neuroprosthetics and motor rehabilitation.

Abstract

Project Jenkins explores how neural activity in the brain can be decoded into robotic movement and, conversely, how movement patterns can be used to generate synthetic neural data. Using real neural data recorded from motor and premotor cortex areas of a macaque monkey named Jenkins, we develop models for decoding (converting brain signals into robotic arm movements) and encoding (simulating brain activity corresponding to a given movement). For the interface between the brain simulation and the physical world, we utilized Koch v1.1 leader and follower robotic arms. We developed an interactive web console that allows users to generate synthetic brain data from joystick movements in real time. Our results are a step towards brain-controlled robotics, prosthetics, and enhancing normal motor function. By accurately modeling brain activity, we take a step toward flexible brain-computer interfaces that generalize beyond predefined movements. To support the research community, we provide open source tools for both synthetic data generation and neural decoding, fostering reproducibility and accelerating progress. The project is available at https://www.808robots.com/projects/jenkins

Figures (5)

• Figure 1: Project Jenkins. Leader arm velocities are computed via forward kinematics, then a transformer generates synthetic neural data. An MLP trained on real monkey neural data decodes it back into velocity space, commanding the follower arm’s movement. (Monkey diagram adapted from kaufman2014cortical; robotic arm images from kochv1-1).
• Figure 2: Our approach in action. The experimenter moves the leader robotic arm, its velocity is recorded and transformed to synthetic neural data using the encoder model. Then, a decoder that was only trained on real neural data is decoding the neural data back into movement velocities, which are passed through inverse kinematics to the follower robotic arm. For the full video, please see the project’s webpage.
• Figure 3: A typical monkey reaching task (adapted from kaufman2014cortical). Jenkins starts with his hand at center, reaching out to one of 8 radial targets.
• Figure 4: The decoding procedure.
• Figure 5: Generating synthetic neural data.