Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lessons from Neuroscience for AI: How integrating Actions, Compositional Structure and Episodic Memory could enable Safe, Interpretable and Human-Like AI

Rajesh P. N. Rao, Vishwas Sathish, Linxing Preston Jiang, Matthew Bryan, Prashant Rangarajan

TL;DR

Foundation models excel at next-token prediction but lack grounding, agency, interpretability, and durable memory. The paper proposes a brain-inspired augmentation that combines hierarchical world models, explicit action/policy modules, and episodic memory, grounded by efference-copy signaling and memory replay. It links predictive coding and active predictive coding (APC) to AI architectures, and details how hierarchical composition and episodic memory can be integrated, including deployment options for coupling world models with LLMs or using external world models. This approach aims to improve safety, interpretability, energy efficiency, and human-like reasoning, while fostering cross-disciplinary collaboration between neuroscience and AI.

Abstract

The phenomenal advances in large language models (LLMs) and other foundation models over the past few years have been based on optimizing large-scale transformer models on the surprisingly simple objective of minimizing next-token prediction loss, a form of predictive coding that is also the backbone of an increasingly popular model of brain function in neuroscience and cognitive science. However, current foundation models ignore three other important components of state-of-the-art predictive coding models: tight integration of actions with generative models, hierarchical compositional structure, and episodic memory. We propose that to achieve safe, interpretable, energy-efficient, and human-like AI, foundation models should integrate actions, at multiple scales of abstraction, with a compositional generative architecture and episodic memory. We present recent evidence from neuroscience and cognitive science on the importance of each of these components. We describe how the addition of these missing components to foundation models could help address some of their current deficiencies: hallucinations and superficial understanding of concepts due to lack of grounding, a missing sense of agency/responsibility due to lack of control, threats to safety and trustworthiness due to lack of interpretability, and energy inefficiency. We compare our proposal to current trends, such as adding chain-of-thought (CoT) reasoning and retrieval-augmented generation (RAG) to foundation models, and discuss new ways of augmenting these models with brain-inspired components. We conclude by arguing that a rekindling of the historically fruitful exchange of ideas between brain science and AI will help pave the way towards safe and interpretable human-centered AI.

Lessons from Neuroscience for AI: How integrating Actions, Compositional Structure and Episodic Memory could enable Safe, Interpretable and Human-Like AI

TL;DR

Foundation models excel at next-token prediction but lack grounding, agency, interpretability, and durable memory. The paper proposes a brain-inspired augmentation that combines hierarchical world models, explicit action/policy modules, and episodic memory, grounded by efference-copy signaling and memory replay. It links predictive coding and active predictive coding (APC) to AI architectures, and details how hierarchical composition and episodic memory can be integrated, including deployment options for coupling world models with LLMs or using external world models. This approach aims to improve safety, interpretability, energy efficiency, and human-like reasoning, while fostering cross-disciplinary collaboration between neuroscience and AI.

Abstract

The phenomenal advances in large language models (LLMs) and other foundation models over the past few years have been based on optimizing large-scale transformer models on the surprisingly simple objective of minimizing next-token prediction loss, a form of predictive coding that is also the backbone of an increasingly popular model of brain function in neuroscience and cognitive science. However, current foundation models ignore three other important components of state-of-the-art predictive coding models: tight integration of actions with generative models, hierarchical compositional structure, and episodic memory. We propose that to achieve safe, interpretable, energy-efficient, and human-like AI, foundation models should integrate actions, at multiple scales of abstraction, with a compositional generative architecture and episodic memory. We present recent evidence from neuroscience and cognitive science on the importance of each of these components. We describe how the addition of these missing components to foundation models could help address some of their current deficiencies: hallucinations and superficial understanding of concepts due to lack of grounding, a missing sense of agency/responsibility due to lack of control, threats to safety and trustworthiness due to lack of interpretability, and energy inefficiency. We compare our proposal to current trends, such as adding chain-of-thought (CoT) reasoning and retrieval-augmented generation (RAG) to foundation models, and discuss new ways of augmenting these models with brain-inspired components. We conclude by arguing that a rekindling of the historically fruitful exchange of ideas between brain science and AI will help pave the way towards safe and interpretable human-centered AI.
Paper Structure (6 sections, 5 figures)

This paper contains 6 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Predictive Coding Theories of Brain Function
  3. Actions, Grounding and World Modeling
  4. Compositional Structure and Hierarchy of States and Actions
  5. Episodic Memory
  6. Discussion and Conclusion

Figures (5)

  • Figure 1: The Hierarchical Brain. (A) Hierarchy in the visual system. (Left) Bidirectionally connected areas of the cortex (in the middle) receiving visual information from the retina (bottom) and storing/receiving episodic context from the hippocampus (at the top). (Right) Simplified version of the cortical hierarchy of visual areas (V1, V2, V4/MT, PIT/MST, etc.). The areas encode increasingly abstract features as we move to the top. (Adapted from Felleman_VanEssen_1991VanEssen_1994) (B) Hierarchy in the motor system. (Left) Hierarchical organization of brain regions implicated in movement. (Right) Abstraction level of processing in these regions. (Adapted from Olveczky-hier2022)
  • Figure 2: Hierarchical Predictive Coding. (A) Hierarchical structure of a predictive coding network (bottom) emulating the hierarchy of visual areas in the cortex (top), with feedback connections carrying predictions of lower-level activity and feedforward connections conveying the errors. (B) Each "Predictive Estimator" module includes feedforward, feedback and locally recurrent synaptic weights that are learned using local prediction errors. These errors are also used during inference to correct the state estimates $\hat{\bf{r}}(t)$ and generate better predictions. (Adapted from Rao_Ballard_1999)
  • Figure 3: Importance of Actions in the Brain and Active Predictive Coding. (A) Almost all areas in the brain, including areas traditionally labeled as “sensory cortex,” are influenced by upcoming actions. The cross sections of the mouse brain on the left (upper row) show neural activation across the entire cortex (measured via widefield calcium imaging) from 0 to 200ms after the onset of a stimulus and before movement in a Left/Right action selection trial ("Go"). In contrast, "NoGo" trials (animal does not perform movement) do not show such activity (lower row). (Right) Average action execution decoder accuracy 25ms prior to movement onset (darker pixels denote higher accuracy). The plot shows that an impending movement can be decoded from cortical activity in most imaged regions. (Adapted from Zatka-Haas2021) (B) Active predictive coding model for a single level. (C) Hierarchical active predictive coding model. See text for details. (From rao2024sensoryrao2023active)
  • Figure 4: A New Brain-Inspired Architecture for Foundation Models. Mod.: modulation of lower network. Eff./Int.: efference copy/internal action for grounding/prediction control. See text for details.
  • Figure 5: Using Compositionality to Tackle Complex Worlds, Tasks and Scenes. (A) Decomposition of the "Go to Grocery Store" problem into sub-goals/tasks, each of which can be further divided into sub-sub-goals/tasks. Note that the rate of change is faster at the lower levels compared to the higher levels, leading naturally to a temporal hierarchy. (B) A navigation problem in a maze-like building environment with corridors (black) and walls (gray), the blue dot indicating current location and green square the desired goal location. The structure of the environment can be understood in terms of its state transition dynamics, which in turn can be divided into the simpler transition dynamics of its compositional elements, two rooms outlined in yellow and red that appear at several different locations within the reference frame of the environment. These simpler elements can be further decomposed into horizontal and vertical corridors shown on the right that appear at different locations within the local reference frame of each room. (C) An object (such as a handwritten digit "8") can be divided into parts (loops and curves at the middle level), each of which can be divided into sub-parts (strokes, lines, smaller curves at the lower level). Each part/sub-part is associated with its coordinates (location/transformation) within a local reference frame. (From rao2024sensory)