AtomMem : Learnable Dynamic Agentic Memory with Atomic Memory Operation

TL;DR

AtomMem reframes agent memory as a dynamic decision problem by decomposing memory management into atomic CRUD operations and learning a policy via reinforcement learning within a semi-Markov framework. A scratchpad plus an external vector memory enable structured, task-aligned memory usage, with a two-stage training pipeline (SFT followed by GRPO) and terminal task rewards guiding policy adaptation. Across HotpotQA, 2WikiMultiHopQA, and MuSiQue, AtomMem-8B outperforms static-memory baselines, with gains increasing as input length grows, and RL shaping memory usage toward selective creation, updating, and deletion while reducing unnecessary reads. The results demonstrate that learnable memory management yields improved long-horizon reasoning and scalable memory control, though at a substantial computational cost for RL training.

Abstract

Equipping agents with memory is essential for solving real-world long-horizon problems. However, most existing agent memory mechanisms rely on static and hand-crafted workflows. This limits the performance and generalization ability of these memory designs, which highlights the need for a more flexible, learning-based memory framework. In this paper, we propose AtomMem, which reframes memory management as a dynamic decision-making problem. We deconstruct high-level memory processes into fundamental atomic CRUD (Create, Read, Update, Delete) operations, transforming the memory workflow into a learnable decision process. By combining supervised fine-tuning with reinforcement learning, AtomMem learns an autonomous, task-aligned policy to orchestrate memory behaviors tailored to specific task demands. Experimental results across 3 long-context benchmarks demonstrate that the trained AtomMem-8B consistently outperforms prior static-workflow memory methods. Further analysis of training dynamics shows that our learning-based formulation enables the agent to discover structured, task-aligned memory management strategies, highlighting a key advantage over predefined routines.

Figures (7)

• Figure 1: The one-size-fits-all workflow of static memory often fails to adapt to diverse tasks. Instead, a dynamic memory system is needed to determine the optimal memory strategy based on the specific task context.
• Figure 2: Overview of the AtomMem framework. The agent interacts with long documents, web, or real-world environments while maintaining an external memory. High-level memory workflows are decomposed into atomic CRUD (Create, Read, Update, Delete) operations over a vector database. Through end-to-end reinforcement learning, the agent learns a task-aligned memory management policy that dynamically decides when to store, retrieve, update, or delete information based on task demands.
• Figure 3: The frequency of the memory operations during the RL training. The y-axis represents the average number of memory API calls made by the model.
• Figure 4: Training curves for optimizing a single component and for jointly optimizing all the components.
• Figure 5: A case illustrates that the model adopts different memory management strategies ($a_t^{mem}$) when facing different task contexts $o_t^{env}$. It demonstrates the dynamic nature of AtomMem .