A Brain-inspired Embodied Intelligence for Fluid and Fast Reflexive Robotics Control

TL;DR

This work introduces NeuroVLA, a brain-inspired, tri-level framework for embodied robotics that decouples semantic planning from fast motor control by distributing computation across cortical (CUDA) and neuromorphic (cerebellar and spinal) tiers. By incorporating a Vision-Language Model backbone with Layer-wise Semantic Distillation (Q-Former), a cerebellar module implementing gated recurrent modulation, and a spiking spinal module with residual dynamics, the approach achieves low-latency reflexes and energy efficiency while retaining open-vocabulary reasoning. Key findings include substantial damping of high-frequency motor noise, emergent motor primitives without extensive supervision, and fast safety reflexes (<20 ms) enabled by local spinal reflex loops, with hardware validation showing 2.19 ms latency and 0.87 mJ per inference on an FPGA-based neuromorphic processor. The results demonstrate robust, energy-efficient embodied intelligence with real-world applicability, highlighting the value of reinstating a biological motor hierarchy for scalable, adaptive robots.

Abstract

Recent advances in embodied intelligence have leveraged massive scaling of data and model parameters to master natural-language command following and multi-task control. In contrast, biological systems demonstrate an innate ability to acquire skills rapidly from sparse experience. Crucially, current robotic policies struggle to replicate the dynamic stability, reflexive responsiveness, and temporal memory inherent in biological motion. Here we present Neuromorphic Vision-Language-Action (NeuroVLA), a framework that mimics the structural organization of the bio-nervous system between the cortex, cerebellum, and spinal cord. We adopt a system-level bio-inspired design: a high-level model plans goals, an adaptive cerebellum module stabilizes motion using high-frequency sensors feedback, and a bio-inspired spinal layer executes lightning-fast actions generation. NeuroVLA represents the first deployment of a neuromorphic VLA on physical robotics, achieving state-of-the-art performance. We observe the emergence of biological motor characteristics without additional data or special guidance: it stops the shaking in robotic arms, saves significant energy(only 0.4w on Neuromorphic Processor), shows temporal memory ability and triggers safety reflexes in less than 20 milliseconds.

Figures (8)

• Figure 1: Hierarchical decoupling of semantic planning and neuromorphic motor control.a, The Bio-inspired Computing Paradigm bridges the timescale gap between cognition and actuation. The architecture allocates high-latency, high-dimensional visual-language processing to a CUDA Computing Tier (Cortical Module), while offloading high-frequency proprioceptive modulation and reflexes to an energy-efficient Neuromorphic Chip Tier (Cerebellar/Spinal Modules). This separation enables a 10$\times$ speedup in local sensorimotor loops compared to cortical planning. b, Data flow in the Tri-Level Neuromorphic VLA. (1) The Cortical Module synthesizes abstract, low-frequency motor goals from visual instructions. (2) The Cerebellar Module functions as a state-adaptive filter, utilizing dense proprioception to perform real-time gain modulation (inset graph), compensating for dynamic discrepancies. (3) The Spiking Spinal Module translates these commands into precise actuation via Spiking Neural Networks (SNNs). Crucially, it incorporates a Fast Safety Reflex pathway (red loop) that processes tactile/force signals locally to trigger withdrawal responses, bypassing the slower cortical loop entirely, while enabling on-device plasticity for continuous adaptation.
• Figure 2: Semantic distillation of descending motor intent via attentional gating. Analogous to the corticospinal tract, which filters high-dimensional cortical processing into streamlined execution commands, the Q-Former interface extracts task-specific geometric features while suppressing task-irrelevant information. a, b, Multi-stage intent extraction for the task "Put the wine bottle on the cabinet." The mechanism initially isolates the manipulation target (wine bottle) in the wrist view to guide grasping (a), before shifting attention to the destination surface to orchestrate placement (b), mirroring the sequential focus of motor planning. c, d, Semantic selectivity under the instruction "Open the middle drawer." Despite the visual salience of the wine bottle (a potential distractor), the descending queries actively inhibit this feature (c), instead exclusively grounding the drawer handle (d). This confirms that the module does not merely encode visual saliency but performs top-down attentional modulation, ensuring that only task-relevant spatial primitives are transmitted to downstream cerebellar and spinal circuits.
• Figure 3: Proprioceptive temporal dynamics and force-aware adaptive control.a, Temporal sensorimotor dynamics as a compact form of motor memory. In rhythmic manipulation tasks (e.g., "Shake the cup"), the system encodes the motion primitive not through redundant visual frames, but via high-frequency trajectories of Right Wrench (force/torque) and Right Arm Joint states. This proprioceptive encoding allows the agent to maintain phase consistency and temporal rhythm independent of visual occlusion. b, Collision-induced wrench perturbations and cerebellum-inspired trajectory adjustment. Real-time monitoring of 6D wrench signals detects a physical contact event (sharp spike in force profiles). Upon detection, the cerebellar feedback loop triggers an immediate spatial trajectory reformulation (blue solid line in 3D plot), allowing the end-effector to autonomously navigate around the obstacle, whereas the open-loop baseline (red dashed line) fails to adapt.
• Figure 4: Cerebellar-mediated attenuation of high-frequency kinematic motor noise.a, Qualitative comparison of commanded acceleration traces over time. The baseline cortical policy (red) exhibits significant high-frequency stochastic jitter (analogous to "intention tremor") across translation ($dx, dy, dz$) and rotation ($d\phi, d\theta, d\psi$) dimensions. The inclusion of the cerebellar module (blue) acts as a physiological damper, producing markedly smoother control signals. b, c, Quantitative assessment of kinematic smoothing. Bar charts show the b, Mean Absolute Commanded Jerk and c, Mean Absolute Commanded Acceleration. The cerebellar module achieves substantial noise attenuation, reducing average jerk by over 75% and average acceleration by over 40% compared to the monolithic baseline, confirming its critical role in stabilizing stochastic cortical outputs prior to spinal execution.
• Figure 5: Event-driven sparsity and temporal robustness of the neuromorphic spinal module.a, b, Selective neural recruitment and functional modularity. a, Traces of membrane potentials and spike trains demonstrate the event-driven nature of the SNN. During the specific phase of "Static Pose & Dynamic Gripper" (shaded region), the Gripper Control Neurons (GCN, red) exhibit high-frequency spiking activity driven by the actuation demand, while the End-Effector Pose Control Neurons (ECN, blue) remain relatively quiescent. b, Quantitative comparison of firing rates confirms this decoupling: neurons are selectively recruited only when their corresponding motor primitives undergo state changes, minimizing redundant computation. c, Metabolic efficiency via temporal sparsity. Mean activation rates across network layers drop significantly during static holding phases compared to dynamic action phases. This "activity-on-demand" mechanism ensures low power consumption, crucial for edge-side deployment on battery-constrained robots. d, Ablation study on the LIBERO benchmark. The Multi-step SNN (blue), which integrates temporal context, consistently outperforms the Single-step SNN (red) and the No-Cerebellum baseline (green). The performance gap is particularly pronounced in long-horizon tasks (e.g., "Bowl on stove"), validating that the spinal module's intrinsic temporal dynamics—analogous to the cerebellum's role in sequencing—are essential for robust complex manipulation.