Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents

TL;DR

The paper addresses the memory bottleneck in long-horizon LLM-based agents by proposing Agentic Memory (AgeMem), a unified framework that jointly manages long-term and short-term memory through a tool-based interface embedded in the agent's policy. It introduces a three-stage progressive RL strategy and a step-wise Group Relative Policy Optimization (GRPO) to align memory decisions with task outcomes across fragmented experiences. Empirical results on five long-context benchmarks show that AgeMem outperforms strong baselines in task performance, memory quality, and context efficiency, demonstrating the value of end-to-end learned memory policies. This work advances scalable, adaptive long-horizon reasoning by tightly integrating memory management into end-to-end decision making for LLM agents, with broad implications for real-world AI assistants.

Abstract

Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows, making effective memory management critical. Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components, relying on heuristics or auxiliary controllers, which limits adaptability and end-to-end optimization. In this paper, we propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy. AgeMem exposes memory operations as tool-based actions, enabling the LLM agent to autonomously decide what and when to store, retrieve, update, summarize, or discard information. To train such unified behaviors, we propose a three-stage progressive reinforcement learning strategy and design a step-wise GRPO to address sparse and discontinuous rewards induced by memory operations. Experiments on five long-horizon benchmarks demonstrate that AgeMem consistently outperforms strong memory-augmented baselines across multiple LLM backbones, achieving improved task performance, higher-quality long-term memory, and more efficient context usage.

Figures (10)

• Figure 1: Comparison between independent and unified memory management frameworks. (Left) Traditional framework with static STM and trigger-based LTM. (Middle) Independent framework with an additional Memory Manager controlling LTM in an agent-based manner, while STM remains static. (Right) The proposed AgeMem framework, where LTM and STM are jointly and intelligently managed via explicit tool-based operations.
• Figure 2: Memory Quality scores for different methods on HotpotQA. Higher scores indicate better relevance between stored memories and ground-truth facts.
• Figure 3: Average prompt token counts under different STM management configurations on HotpotQA. The suffix "-RAG" indicates the adoption of RAG in place of STM tool-based management.
• Figure 4: Ablation study on LTM, STM, and RL components (Qwen2.5-7B-Instruct). Base: No-memory baseline; +LT: AgeMem-noRL-RAG (LTM tools only); +LT/RL: AgeMem-RAG (RL with LTM tools); +LT/ST/RL: AgeMem (full AgeMem system with RL). Green arrows indicate performance gains over the baseline.
• Figure 5: Training convergence curves on Qwen2.5-7B-Instruct comparing All-Returns (solid line) v.s. Answer-Only (dashed line) reward strategies.