Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory

TL;DR

This work targets the gap that LLM memory is predominantly static and focused on recall rather than experiential reuse. It introduces Evo-Memory, a streaming benchmark that reframes static datasets as sequential task streams and formalizes memory-augmented agents as a loop over retrieval, synthesis, and memory evolution, represented by the tuple $(F,U,R,C)$ and the memory state $M_t$. Two instantiations, ExpRAG and ReMem, instantiate retrieval-based experience reuse and an integrated thinking-action-memory refinement loop, respectively. Evaluated across 10 diverse datasets with backbones Gemini-2.5 and Claude, Evo-Memory shows that self-evolving memory improves accuracy, efficiency, and robustness, especially in multi-turn embodied tasks, and provides a reproducible protocol and insights into memory organization, pruning, and transfer for future adaptive LLM agents.

Abstract

Statefulness is essential for large language model (LLM) agents to perform long-term planning and problem-solving. This makes memory a critical component, yet its management and evolution remain largely underexplored. Existing evaluations mostly focus on static conversational settings, where memory is passively retrieved from dialogue to answer queries, overlooking the dynamic ability to accumulate and reuse experience across evolving task streams. In real-world environments such as interactive problem assistants or embodied agents, LLMs are required to handle continuous task streams, yet often fail to learn from accumulated interactions, losing valuable contextual insights, a limitation that calls for test-time evolution, where LLMs retrieve, integrate, and update memory continuously during deployment. To bridge this gap, we introduce Evo-Memory, a comprehensive streaming benchmark and framework for evaluating self-evolving memory in LLM agents. Evo-Memory structures datasets into sequential task streams, requiring LLMs to search, adapt, and evolve memory after each interaction. We unify and implement over ten representative memory modules and evaluate them across 10 diverse multi-turn goal-oriented and single-turn reasoning and QA datasets. To better benchmark experience reuse, we provide a baseline method, ExpRAG, for retrieving and utilizing prior experience, and further propose ReMem, an action-think-memory refine pipeline that tightly integrates reasoning, task actions, and memory updates to achieve continual improvement.

Figures (8)

• Figure 1: Conversational recall retrieves past facts (e.g., solutions to $2x^2 + 3x - 1 = 0$). Experience reuse recalls reasoning strategies (e.g., using the formula).
• Figure 2: Illustration of different task types and experience reusing. A stateful agent encounters both multi-turn tasks (e.g., embodied manipulation) and single-turn tasks (e.g., solving equations), and should learn reusable experiences from past experiences.
• Figure 3: Overview of the ReMem agent framework. Left: Test-time evolution process where the agent iteratively searches, synthesizes, and evolves its memory across multiple tasks. Right: Agent architecture with three core modules—Think (reasoning and decomposition), Refine Memory (retrieve, prune, organize), and Act (execution)—that interact with the environment and learned memory.
• Figure 4: ReMem performance gain over history baseline versus within-dataset task similarity.
• Figure 5: Average steps to complete tasks across four benchmarks. We compare four methods: History, ExpRecent, ExpRAG, and ReMem. Lower is better. ReMem consistently requires fewer steps to complete tasks across all datasets, demonstrating more efficient task execution..