Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning How to Remember: A Meta-Cognitive Management Method for Structured and Transferable Agent Memory

Sirui Liang, Pengfei Cao, Jian Zhao, Wenhao Teng, Xiangwen Liao, Jun Zhao, Kang Liu

TL;DR

The Meta-Cognitive Memory Abstraction method (MCMA), which treats memory abstraction as a learnable cognitive skill rather than a fixed design choice, is proposed, which decouples task execution from memory management by combining a frozen task model with a learned memory copilot.

Abstract

Large language model (LLM) agents increasingly rely on accumulated memory to solve long-horizon decision-making tasks. However, most existing approaches store memory in fixed representations and reuse it at a single or implicit level of abstraction, which limits generalization and often leads to negative transfer when distribution shift. This paper proposes the Meta-Cognitive Memory Abstraction method (MCMA), which treats memory abstraction as a learnable cognitive skill rather than a fixed design choice. MCMA decouples task execution from memory management by combining a frozen task model with a learned memory copilot. The memory copilot is trained using direct preference optimization, it determines how memories should be structured, abstracted, and reused. Memories are further organized into a hierarchy of abstraction levels, enabling selective reuse based on task similarity. When no memory is transferable, MCMA transfers the ability to abstract and manage memory by transferring the memory copilot. Experiments on ALFWorld, ScienceWorld, and BabyAI demonstrate substantial improvements in performance, out-of-distribution generalization, and cross-task transfer over several baselines.

Learning How to Remember: A Meta-Cognitive Management Method for Structured and Transferable Agent Memory

TL;DR

The Meta-Cognitive Memory Abstraction method (MCMA), which treats memory abstraction as a learnable cognitive skill rather than a fixed design choice, is proposed, which decouples task execution from memory management by combining a frozen task model with a learned memory copilot.

Abstract

Large language model (LLM) agents increasingly rely on accumulated memory to solve long-horizon decision-making tasks. However, most existing approaches store memory in fixed representations and reuse it at a single or implicit level of abstraction, which limits generalization and often leads to negative transfer when distribution shift. This paper proposes the Meta-Cognitive Memory Abstraction method (MCMA), which treats memory abstraction as a learnable cognitive skill rather than a fixed design choice. MCMA decouples task execution from memory management by combining a frozen task model with a learned memory copilot. The memory copilot is trained using direct preference optimization, it determines how memories should be structured, abstracted, and reused. Memories are further organized into a hierarchy of abstraction levels, enabling selective reuse based on task similarity. When no memory is transferable, MCMA transfers the ability to abstract and manage memory by transferring the memory copilot. Experiments on ALFWorld, ScienceWorld, and BabyAI demonstrate substantial improvements in performance, out-of-distribution generalization, and cross-task transfer over several baselines.
Paper Structure (36 sections, 10 equations, 20 figures, 8 tables, 1 algorithm)

This paper contains 36 sections, 10 equations, 20 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Memory as Content Retrieval or Execution Guidance.
  4. Hierarchical, Reflective, and Learned Experience
  5. Methodology
  6. Trajectory Collection and Preprocessing
  7. Multi-Structure Memory Generation
  8. Memory Copilot Evolution
  9. Hierarchical Abstraction and Cross-Task Reusing
  10. Experiment
  11. Experimental Setup
  12. Datasets.
  13. Metrics.
  14. Models.
  15. Baselines.
  16. ...and 21 more sections

Figures (20)

  • Figure 1: An example of how MCMA works.
  • Figure 2: The task model collects raw trajectories, which the memory copilot abstracts into structured knowledge. Preference pairs constructed from downstream task performance are used to train the memory copilot via DPO. Learned memories and the copilot are organized hierarchically to support adaptive memory reuse across tasks.
  • Figure 3: The impact of the number of knowledge provided by MCMA in the ALFWorld task.
  • Figure 4: Analyze knowledge structure of MCMA.
  • Figure 5: A simplified trajectory of an agent performing an Examine in Light task in ALFWorld.
  • ...and 15 more figures