Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diagnosing Retrieval vs. Utilization Bottlenecks in LLM Agent Memory

Boqin Yuan, Yue Su, Kun Yao

TL;DR

It is argued that, under current retrieval practices, improving retrieval quality yields larger gains than increasing write-time sophistication more than increasing write-time sophistication.

Abstract

Memory-augmented LLM agents store and retrieve information from prior interactions, yet the relative importance of how memories are written versus how they are retrieved remains unclear. We introduce a diagnostic framework that analyzes how performance differences manifest across write strategies, retrieval methods, and memory utilization behavior, and apply it to a 3x3 study crossing three write strategies (raw chunks, Mem0-style fact extraction, MemGPT-style summarization) with three retrieval methods (cosine, BM25, hybrid reranking). On LoCoMo, retrieval method is the dominant factor: average accuracy spans 20 points across retrieval methods (57.1% to 77.2%) but only 3-8 points across write strategies. Raw chunked storage, which requires zero LLM calls, matches or outperforms expensive lossy alternatives, suggesting that current memory pipelines may discard useful context that downstream retrieval mechanisms fail to compensate for. Failure analysis shows that performance breakdowns most often manifest at the retrieval stage rather than at utilization. We argue that, under current retrieval practices, improving retrieval quality yields larger gains than increasing write-time sophistication. Code is publicly available at https://github.com/boqiny/memory-probe.

Diagnosing Retrieval vs. Utilization Bottlenecks in LLM Agent Memory

TL;DR

It is argued that, under current retrieval practices, improving retrieval quality yields larger gains than increasing write-time sophistication more than increasing write-time sophistication.

Abstract

Memory-augmented LLM agents store and retrieve information from prior interactions, yet the relative importance of how memories are written versus how they are retrieved remains unclear. We introduce a diagnostic framework that analyzes how performance differences manifest across write strategies, retrieval methods, and memory utilization behavior, and apply it to a 3x3 study crossing three write strategies (raw chunks, Mem0-style fact extraction, MemGPT-style summarization) with three retrieval methods (cosine, BM25, hybrid reranking). On LoCoMo, retrieval method is the dominant factor: average accuracy spans 20 points across retrieval methods (57.1% to 77.2%) but only 3-8 points across write strategies. Raw chunked storage, which requires zero LLM calls, matches or outperforms expensive lossy alternatives, suggesting that current memory pipelines may discard useful context that downstream retrieval mechanisms fail to compensate for. Failure analysis shows that performance breakdowns most often manifest at the retrieval stage rather than at utilization. We argue that, under current retrieval practices, improving retrieval quality yields larger gains than increasing write-time sophistication. Code is publicly available at https://github.com/boqiny/memory-probe.
Paper Structure (25 sections, 4 figures, 5 tables)

This paper contains 25 sections, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Write Strategies
  4. Retrieval Methods
  5. Probing Framework
  6. Results
  7. Write Strategy Has Minimal Impact
  8. Retrieval Method Dominates
  9. Failure Analysis Confirms Retrieval as the Bottleneck
  10. Discussion and Conclusion
  11. Limitations.
  12. Related Work
  13. Experimental Design and Rationale
  14. Dataset Details.
  15. Memory Prompts
  16. ...and 10 more sections

Figures (4)

  • Figure 1: Memory-agent pipeline with three diagnostic probes at the retrieval-to-generation boundary.
  • Figure 2: Accuracy across the $3 \times 3$ grid ($k{=}5$). Retrieval method drives 14--23 point differences; write strategy only 3--8.
  • Figure 3: Precision@5 vs. accuracy ($r{=}0.98$). Each point is one of nine configurations.
  • Figure 4: Retrieval failure rate by retrieval method. Hybrid reranking cuts retrieval failures by half or more across all write strategies.