Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Memory OS of AI Agent

Jiazheng Kang, Mingming Ji, Zhe Zhao, Ting Bai

TL;DR

The paper tackles the challenge of long-term memory in LLM agents caused by fixed context windows. It introduces MemoryOS, an OS-inspired framework that organizes memory into short-term, mid-term, and long-term personal layers, with four coordinated modules for storage, updating, retrieval, and generation. MemoryOS employs FIFO-based STM→MTM updates and heat-driven segmented paging MTM→LPM, enabling efficient memory upkeep and personalized persona evolution, validated on LoCoMo and GVD benchmarks with significant gains in F1 and BLEU-1. The work demonstrates that hierarchical memory management and adaptive retrieval can sustain coherent, personalized interactions over extended conversations, and it provides open-source implementation for broader adoption.

Abstract

Large Language Models (LLMs) face a crucial challenge from fixed context windows and inadequate memory management, leading to a severe shortage of long-term memory capabilities and limited personalization in the interactive experience with AI agents. To overcome this challenge, we innovatively propose a Memory Operating System, i.e., MemoryOS, to achieve comprehensive and efficient memory management for AI agents. Inspired by the memory management principles in operating systems, MemoryOS designs a hierarchical storage architecture and consists of four key modules: Memory Storage, Updating, Retrieval, and Generation. Specifically, the architecture comprises three levels of storage units: short-term memory, mid-term memory, and long-term personal memory. Key operations within MemoryOS include dynamic updates between storage units: short-term to mid-term updates follow a dialogue-chain-based FIFO principle, while mid-term to long-term updates use a segmented page organization strategy. Our pioneering MemoryOS enables hierarchical memory integration and dynamic updating. Extensive experiments on the LoCoMo benchmark show an average improvement of 49.11% on F1 and 46.18% on BLEU-1 over the baselines on GPT-4o-mini, showing contextual coherence and personalized memory retention in long conversations. The implementation code is open-sourced at https://github.com/BAI-LAB/MemoryOS.

Memory OS of AI Agent

TL;DR

The paper tackles the challenge of long-term memory in LLM agents caused by fixed context windows. It introduces MemoryOS, an OS-inspired framework that organizes memory into short-term, mid-term, and long-term personal layers, with four coordinated modules for storage, updating, retrieval, and generation. MemoryOS employs FIFO-based STM→MTM updates and heat-driven segmented paging MTM→LPM, enabling efficient memory upkeep and personalized persona evolution, validated on LoCoMo and GVD benchmarks with significant gains in F1 and BLEU-1. The work demonstrates that hierarchical memory management and adaptive retrieval can sustain coherent, personalized interactions over extended conversations, and it provides open-source implementation for broader adoption.

Abstract

Large Language Models (LLMs) face a crucial challenge from fixed context windows and inadequate memory management, leading to a severe shortage of long-term memory capabilities and limited personalization in the interactive experience with AI agents. To overcome this challenge, we innovatively propose a Memory Operating System, i.e., MemoryOS, to achieve comprehensive and efficient memory management for AI agents. Inspired by the memory management principles in operating systems, MemoryOS designs a hierarchical storage architecture and consists of four key modules: Memory Storage, Updating, Retrieval, and Generation. Specifically, the architecture comprises three levels of storage units: short-term memory, mid-term memory, and long-term personal memory. Key operations within MemoryOS include dynamic updates between storage units: short-term to mid-term updates follow a dialogue-chain-based FIFO principle, while mid-term to long-term updates use a segmented page organization strategy. Our pioneering MemoryOS enables hierarchical memory integration and dynamic updating. Extensive experiments on the LoCoMo benchmark show an average improvement of 49.11% on F1 and 46.18% on BLEU-1 over the baselines on GPT-4o-mini, showing contextual coherence and personalized memory retention in long conversations. The implementation code is open-sourced at https://github.com/BAI-LAB/MemoryOS.

Paper Structure

This paper contains 40 sections, 5 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Memory for LLM Agents
  4. Memory Management in OS
  5. MemoryOS
  6. Overview Architecture
  7. Memory Storage:
  8. Memory Updating:
  9. Memory Retrieval:
  10. Response Generation:
  11. Memory Storage Module
  12. Short-Term Memory (STM):
  13. Mid-Term Memory (MTM):
  14. Long-term Persona Memory (LPM):
  15. Memory Update Module
  16. ...and 25 more sections

Figures (4)

  • Figure 1: The overview architecture of MemoryOS, including memory Store, Updating, Retrieval, Response.
  • Figure 2: The ablation study on the GVD and LoCoMo benchmark datasets.
  • Figure 3: Impact of hyperparameter $k$ (retrieved pages in MTM) on LoCoMo benchmark.
  • Figure 4: Case study demonstrating the positive impact of introducing our memory management system. Left: default LLMs; Right: with MemoryOS.