Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agents

Jiafeng Liang, Hao Li, Chang Li, Jiaqi Zhou, Shixin Jiang, Zekun Wang, Changkai Ji, Zhihao Zhu, Runxuan Liu, Tao Ren, Jinlan Fu, See-Kiong Ng, Xia Liang, Ming Liu, Bing Qin

TL;DR

The paper addresses the challenge of endowing autonomous agents with memory capabilities by bridging cognitive neuroscience and AI memory systems. It presents a progressive brain-to-LLM-to-agent view, offering a unified taxonomy and lifecycle for memory that spans definitions, storage, management, benchmarks, and security. Key contributions include correlating neuroscience memory types (short-term vs long-term) with agent memory (episodic vs semantic, inside-trail vs cross-trail), detailing storage formats and locations, outlining closed-loop memory management, and surveying semantic- and episodic-oriented benchmarks along with security considerations. The work highlights future directions in multimodal memory and cross-agent skill sharing, aiming to build memory-enabled, personalized, and secure agents capable of long-horizon reasoning and continual adaptation.

Abstract

Memory serves as the pivotal nexus bridging past and future, providing both humans and AI systems with invaluable concepts and experience to navigate complex tasks. Recent research on autonomous agents has increasingly focused on designing efficient memory workflows by drawing on cognitive neuroscience. However, constrained by interdisciplinary barriers, existing works struggle to assimilate the essence of human memory mechanisms. To bridge this gap, we systematically synthesizes interdisciplinary knowledge of memory, connecting insights from cognitive neuroscience with LLM-driven agents. Specifically, we first elucidate the definition and function of memory along a progressive trajectory from cognitive neuroscience through LLMs to agents. We then provide a comparative analysis of memory taxonomy, storage mechanisms, and the complete management lifecycle from both biological and artificial perspectives. Subsequently, we review the mainstream benchmarks for evaluating agent memory. Additionally, we explore memory security from dual perspectives of attack and defense. Finally, we envision future research directions, with a focus on multimodal memory systems and skill acquisition.

AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agents

TL;DR

The paper addresses the challenge of endowing autonomous agents with memory capabilities by bridging cognitive neuroscience and AI memory systems. It presents a progressive brain-to-LLM-to-agent view, offering a unified taxonomy and lifecycle for memory that spans definitions, storage, management, benchmarks, and security. Key contributions include correlating neuroscience memory types (short-term vs long-term) with agent memory (episodic vs semantic, inside-trail vs cross-trail), detailing storage formats and locations, outlining closed-loop memory management, and surveying semantic- and episodic-oriented benchmarks along with security considerations. The work highlights future directions in multimodal memory and cross-agent skill sharing, aiming to build memory-enabled, personalized, and secure agents capable of long-horizon reasoning and continual adaptation.

Abstract

Memory serves as the pivotal nexus bridging past and future, providing both humans and AI systems with invaluable concepts and experience to navigate complex tasks. Recent research on autonomous agents has increasingly focused on designing efficient memory workflows by drawing on cognitive neuroscience. However, constrained by interdisciplinary barriers, existing works struggle to assimilate the essence of human memory mechanisms. To bridge this gap, we systematically synthesizes interdisciplinary knowledge of memory, connecting insights from cognitive neuroscience with LLM-driven agents. Specifically, we first elucidate the definition and function of memory along a progressive trajectory from cognitive neuroscience through LLMs to agents. We then provide a comparative analysis of memory taxonomy, storage mechanisms, and the complete management lifecycle from both biological and artificial perspectives. Subsequently, we review the mainstream benchmarks for evaluating agent memory. Additionally, we explore memory security from dual perspectives of attack and defense. Finally, we envision future research directions, with a focus on multimodal memory systems and skill acquisition.
Paper Structure (62 sections, 5 figures, 2 tables)

This paper contains 62 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Memory Definitions
  3. Memory from the Perspective of Cognitive Neuroscience
  4. Memory from the Perspective of Large Language Models
  5. Parametric Memory
  6. Working Memory
  7. Explicit External Memory
  8. Memory from the Perspective of Agents
  9. Structured Storage
  10. Dynamic Scheduling Mechanisms
  11. Cognitive Processing and Evolution
  12. The Difference Between Agent Memory and RAG
  13. Memory Utility
  14. Breaking Context Window Constraints
  15. Heuristic Context Design
  16. ...and 47 more sections

Figures (5)

  • Figure 1: Overview of memory utility in LLM-driven agents. Memory extends agent capabilities by alleviating context window constraints, enabling long-term personalization, and driving experience-based reasoning through a feedback loop with reflection and planning.
  • Figure 2: Overview of the memory classification in agents. (a) Nature-based taxonomy that categorizes memory based on the type of information being encoded. (b) Scope-based classification that distinguishes memory according to how broadly it can be applied.
  • Figure 3: Overview of memory storage mechanisms in cognitive neuroscience, including storage locations and storage formats of short- and long-term memory.
  • Figure 4: Overview of memory management in cognitive neuroscience. The framework illustrates a dynamic cycle of information processing including memory formation, updating, and retrieval, through which long-term memory supports flexible adaptation to the external environment.
  • Figure 5: Overview of memory management in agents. The framework forms a closed-loop pipeline consisting of memory extraction, updating, retrieval, and utilization, enabling persistent experience regulation and long-range reasoning.