Neural Brain: A Neuroscience-inspired Framework for Embodied Agents

TL;DR

This work proposes Neural Brain, a neuroscience-inspired framework for embodied agents that unifies multimodal sensing, closed-loop perception-cognition-action, neuroplastic memory, and energy-efficient neuromorphic hardware-software co-design. Grounded in human brain principles such as predictive coding, hierarchical memory, and sparse, event-driven computation, the framework aims to bridge static AI models and the dynamic adaptability required for real-world deployment. It provides design principles, a literature synthesis across sensing, cognition, memory, and hardware, and a roadmap highlighting open challenges and future directions toward generalizable, autonomous embodied intelligence. The approach envisions robust, low-power embodied agents capable of interacting with unstructured environments, leveraging edge computing and neuromorphic computing to sustain real-time operation and continual learning.

Abstract

The rapid evolution of artificial intelligence (AI) has shifted from static, data-driven models to dynamic systems capable of perceiving and interacting with real-world environments. Despite advancements in pattern recognition and symbolic reasoning, current AI systems, such as large language models, remain disembodied, unable to physically engage with the world. This limitation has driven the rise of embodied AI, where autonomous agents, such as humanoid robots, must navigate and manipulate unstructured environments with human-like adaptability. At the core of this challenge lies the concept of Neural Brain, a central intelligence system designed to drive embodied agents with human-like adaptability. A Neural Brain must seamlessly integrate multimodal sensing and perception with cognitive capabilities. Achieving this also requires an adaptive memory system and energy-efficient hardware-software co-design, enabling real-time action in dynamic environments. This paper introduces a unified framework for the Neural Brain of embodied agents, addressing two fundamental challenges: (1) defining the core components of Neural Brain and (2) bridging the gap between static AI models and the dynamic adaptability required for real-world deployment. To this end, we propose a biologically inspired architecture that integrates multimodal active sensing, perception-cognition-action function, neuroplasticity-based memory storage and updating, and neuromorphic hardware/software optimization. Furthermore, we also review the latest research on embodied agents across these four aspects and analyze the gap between current AI systems and human intelligence. By synthesizing insights from neuroscience, we outline a roadmap towards the development of generalizable, autonomous agents capable of human-level intelligence in real-world scenarios.

Figures (17)

• Figure 1: Human brain-inspired Neural Brain. The human brain comprises four key components: sensing, function (perception, cognition, action), memory (short-term and long-term), and implementation features, such as sparse activation, event-driven processing, predictive coding, and distributed and parallel mechanisms. Inspired by insights from neuroscience, we propose the concept of a Neural Brain for Embodied Agents, which integrates these principles into four distinct modules. The sensing module incorporates multimodal fusion, active sensing, and adaptive calibration to enhance perceptual capabilities. The function module encompasses predictive perception, cognitive reasoning, and action, including an action-closed loop to ensure continuous interaction with the environment. The memory module features a hierarchical architecture, neuroplastic adaptation, and context awareness, enabling agents to store and retrieve information dynamically and efficiently. Finally, the hardware/software module is characterized by event-driven processing, neuromorphic architecture, and hardware-software co-design, ensuring robust and flexible operation. These four core ideas, derived from the structure and functionality of the human brain, aim to empower embodied agents to adapt, learn, and perform effectively in real-world, embodied environments.
• Figure 2: The evolution from AI to embodied AI. (a) AI excels in pattern recognition but lacks physical interaction with the real world. (b) Embodied AI enables robots like Atlas of Boston Dynamics and Unitree G1 to perceive and act in their environment. (c) Inspired by the human brain, intelligence arises from neural processes that integrate sensing, perception, cognition, action, and memory. (d) Furthermore, this work proposes a concept of "Neural Brain" for embodied agents, combining neuroscience to achieve generalizable embodied AI.
• Figure 3: A schematic overview illustrating how bottom-up sensing and top-down attention contribute to perception, with bottom-up processes driven by sensory input and top-down mechanisms modulating attention--either by amplifying or suppressing specific stimuli--based on task goals.
• Figure 4: Schematic illustration of the closed-loop framework of multimodal perception, cognition, and action. The process begins with sensing-driven perception--integrating multimodal sensory inputs such as vision, audition, space, and time. These perceptions are processed through cognitive mechanisms to infer context and determine appropriate responses, and resulting actions dynamically influence subsequent perceptual input.
• Figure 5: Schematic representation of efficient memory storage and update, where information enters through short-term and working memory and is transferred to the hippocampus via repetition and rehearsal. The hippocampus encodes and retrieves information through a hashing-like mechanism and consolidates it via memory replay. Long-term memory is refined through neuronal competition, task-dependent adaptive resolution, and forgetting via synaptic weakening.