MemR$^3$: Memory Retrieval via Reflective Reasoning for LLM Agents

TL;DR

MemR$^3$ tackles the challenge of memory retrieval for long-horizon LLM agents by reframing retrieval as a closed-loop, autonomous process. It introduces a Router that alternates between retrieval, reflective reasoning, and answer generation, and a Global Evidence-Gap Tracker that explicitly tracks known evidence and outstanding gaps. The system, built on LangGraph and compatible with existing memory backends, yields consistent gains on the LoCoMo benchmark across RAG and Zep backends, approaching Full-Context in several settings and offering enhanced transparency and efficiency. Ablation analyses and protocol realignments further demonstrate the value of explicit evidence management and iterative query refinement for robust, explainable memory-enabled reasoning.

Abstract

Memory systems have been designed to leverage past experiences in Large Language Model (LLM) agents. However, many deployed memory systems primarily optimize compression and storage, with comparatively less emphasis on explicit, closed-loop control of memory retrieval. From this observation, we build memory retrieval as an autonomous, accurate, and compatible agent system, named MemR$^3$, which has two core mechanisms: 1) a router that selects among retrieve, reflect, and answer actions to optimize answer quality; 2) a global evidence-gap tracker that explicitly renders the answering process transparent and tracks the evidence collection process. This design departs from the standard retrieve-then-answer pipeline by introducing a closed-loop control mechanism that enables autonomous decision-making. Empirical results on the LoCoMo benchmark demonstrate that MemR$^3$ surpasses strong baselines on LLM-as-a-Judge score, and particularly, it improves existing retrievers across four categories with an overall improvement on RAG (+7.29%) and Zep (+1.94%) using GPT-4.1-mini backend, offering a plug-and-play controller for existing memory stores.

Figures (6)

• Figure 1: Illustration of three memory-usage paradigms. Full-Context overloads the LLM with all memories and answers incorrectly; Retrieve-then-Answer retrieves relevant snippets but still miscalculates. In contrast, MemR$^3$ iteratively retrieves and reflects using an evidence–gap tracker (Acts 0–3), refines the query about Buddy’s adoption date, and produces the correct answer (3 months).
• Figure 2: Pipeline of MemR$^3$. MemR$^3$ transforms retrieval into a closed-loop process: a router dynamically switches between Retrieve, Reflect, and Answer nodes while a global evidence–gap tracker maintains what is known and what is still missing. This enables iterative query refinement, targeted retrieval, and early stopping, making MemR$^3$ an autonomous, backend-agnostic retrieval controller.
• Figure 3: Example of the evidence-gap tracker for a specific query. At each step, the agent maintains an explicit summary of the evidence established and the information still missing. This state can be presented directly to users as a human-readable explanation of the agent’s progress in answering the query.
• Figure 4: LLM-as-a-Judge score (%) with different a) number of chunks per iteration and b) max iterations.
• Figure 5: Number of questions requiring different numbers of iterations before final answers, across four categories.

Theorems & Definitions (6)

• Theorem 3.1: [Informal] Monotonicity, soundness, and completeness of the idealized evidence-gap tracker
• Definition 2.1: Idealized Requirement Space
• Definition 2.2: Memory-Support Relation
• Definition 2.3: Idealized Evidence-Gap Update Rule
• Theorem 2.4: Properties of the Idealized Tracker
• proof