Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating Long-Term Memory for Long-Context Question Answering

Alessandra Terranova, Björn Ross, Alexandra Birch

TL;DR

The paper investigates how memory components—semantic, episodic, and procedural—affect long-context question answering. It benchmarks memory-augmented architectures (RAG, A-Mem, PromptOpt, EpMem) against a Full Context baseline on LoCoMo and, to some extent, QMSum, revealing substantial token-efficiency gains with competitive accuracy. Key findings show RAG is particularly effective for foundation models, while episodic memory aids metacognitive capabilities in instruction-tuned models; procedural memory can underperform for QA but may help transfer in other tasks. The work provides design guidance for deploying memory-augmented LLM agents in knowledge-intensive, long-context settings and outlines directions for adaptive, selective memory mechanisms and future research comparisons with RL-based approaches.

Abstract

In order for large language models to achieve true conversational continuity and benefit from experiential learning, they need memory. While research has focused on the development of complex memory systems, it remains unclear which types of memory are most effective for long-context conversational tasks. We present a systematic evaluation of memory-augmented methods on long-context dialogues annotated for question-answering tasks that require diverse reasoning strategies. We analyse full-context prompting, semantic memory through retrieval-augmented generation and agentic memory, episodic memory through in-context learning, and procedural memory through prompt optimization. Our findings show that memory-augmented approaches reduce token usage by over 90\% while maintaining competitive accuracy. Memory architecture complexity should scale with model capability, with foundation models benefitting most from RAG, and stronger instruction-tuned models gaining from episodic learning through reflections and more complex agentic semantic memory. In particular, episodic memory can help LLMs recognise the limits of their own knowledge.

Evaluating Long-Term Memory for Long-Context Question Answering

TL;DR

The paper investigates how memory components—semantic, episodic, and procedural—affect long-context question answering. It benchmarks memory-augmented architectures (RAG, A-Mem, PromptOpt, EpMem) against a Full Context baseline on LoCoMo and, to some extent, QMSum, revealing substantial token-efficiency gains with competitive accuracy. Key findings show RAG is particularly effective for foundation models, while episodic memory aids metacognitive capabilities in instruction-tuned models; procedural memory can underperform for QA but may help transfer in other tasks. The work provides design guidance for deploying memory-augmented LLM agents in knowledge-intensive, long-context settings and outlines directions for adaptive, selective memory mechanisms and future research comparisons with RL-based approaches.

Abstract

In order for large language models to achieve true conversational continuity and benefit from experiential learning, they need memory. While research has focused on the development of complex memory systems, it remains unclear which types of memory are most effective for long-context conversational tasks. We present a systematic evaluation of memory-augmented methods on long-context dialogues annotated for question-answering tasks that require diverse reasoning strategies. We analyse full-context prompting, semantic memory through retrieval-augmented generation and agentic memory, episodic memory through in-context learning, and procedural memory through prompt optimization. Our findings show that memory-augmented approaches reduce token usage by over 90\% while maintaining competitive accuracy. Memory architecture complexity should scale with model capability, with foundation models benefitting most from RAG, and stronger instruction-tuned models gaining from episodic learning through reflections and more complex agentic semantic memory. In particular, episodic memory can help LLMs recognise the limits of their own knowledge.

Paper Structure

This paper contains 38 sections, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Memory for LLM agents
  4. Retrieval Augmented Generation
  5. Learning through memory
  6. Methodologies
  7. Dataset and evaluation
  8. Memory for Question Answering
  9. Full Context Prompting
  10. RAG: Semantic Memory as Retrieval-Augmented Generation
  11. A-Mem: Agentic Semantic Memory
  12. PromptOpt: Procedural Memory Through Prompt Optimization
  13. EpMem: Episodic Memory Through In-Context Learning
  14. Implementation details
  15. Experiments and Results
  16. ...and 23 more sections

Figures (3)

  • Figure 1: Left figure: EpMem, episodic memory through in-context learning. The model generates answers for a sample of questions and reflects on its performance using F1 scores and labelled answers to produce natural language reflections. These previous examples and reflections are stored in episodic memory and retrieved as in-context examples. Right figure: PromptOpt, procedural memory through prompt optimization. The model answers a sample of questions using an initial prompt and predictions are compared with labelled answers to compute F1 scores. A classification and optimization step is then used to update the prompt, forming procedural memory.
  • Figure 2: Example of episodic memory generated by GPT-4o mini
  • Figure 3: Example of adversarial question