Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

E-mem: Multi-agent based Episodic Context Reconstruction for LLM Agent Memory

Kaixiang Wang, Yidan Lin, Jiong Lou, Zhaojiacheng Zhou, Bunyod Suvonov, Jie Li

TL;DR

E-mem introduces a heterogeneous Master-Assistant architecture that preserves uncompressed episodic memory through Episodic Context Reconstruction to enable rigorous System 2-like reasoning over long horizons. A multi-pathway routing mechanism selectively activates memory segments, while assistant agents locally re-experience and reason within full contexts before the master aggregates evidence, achieving state-of-the-art performance on LoCoMo and HotpotQA with substantial token-cost savings. The approach addresses destructive de-contextualization inherent in memory preprocessing and demonstrates robustness to adversarial inputs, with ablations informing optimal memory chunking and routing design. While the method incurs higher latency during reasoning, it offers a favorable latency-fidelity trade-off for high-precision, knowledge-intensive tasks in domains such as legal, medical, and scientific contexts.

Abstract

The evolution of Large Language Model (LLM) agents towards System~2 reasoning, characterized by deliberative, high-precision problem-solving, requires maintaining rigorous logical integrity over extended horizons. However, prevalent memory preprocessing paradigms suffer from destructive de-contextualization. By compressing complex sequential dependencies into pre-defined structures (e.g., embeddings or graphs), these methods sever the contextual integrity essential for deep reasoning. To address this, we propose E-mem, a framework shifting from Memory Preprocessing to Episodic Context Reconstruction. Inspired by biological engrams, E-mem employs a heterogeneous hierarchical architecture where multiple assistant agents maintain uncompressed memory contexts, while a central master agent orchestrates global planning. Unlike passive retrieval, our mechanism empowers assistants to locally reason within activated segments, extracting context-aware evidence before aggregation. Evaluations on the LoCoMo benchmark demonstrate that E-mem achieves over 54\% F1, surpassing the state-of-the-art GAM by 7.75\%, while reducing token cost by over 70\%.

E-mem: Multi-agent based Episodic Context Reconstruction for LLM Agent Memory

TL;DR

E-mem introduces a heterogeneous Master-Assistant architecture that preserves uncompressed episodic memory through Episodic Context Reconstruction to enable rigorous System 2-like reasoning over long horizons. A multi-pathway routing mechanism selectively activates memory segments, while assistant agents locally re-experience and reason within full contexts before the master aggregates evidence, achieving state-of-the-art performance on LoCoMo and HotpotQA with substantial token-cost savings. The approach addresses destructive de-contextualization inherent in memory preprocessing and demonstrates robustness to adversarial inputs, with ablations informing optimal memory chunking and routing design. While the method incurs higher latency during reasoning, it offers a favorable latency-fidelity trade-off for high-precision, knowledge-intensive tasks in domains such as legal, medical, and scientific contexts.

Abstract

The evolution of Large Language Model (LLM) agents towards System~2 reasoning, characterized by deliberative, high-precision problem-solving, requires maintaining rigorous logical integrity over extended horizons. However, prevalent memory preprocessing paradigms suffer from destructive de-contextualization. By compressing complex sequential dependencies into pre-defined structures (e.g., embeddings or graphs), these methods sever the contextual integrity essential for deep reasoning. To address this, we propose E-mem, a framework shifting from Memory Preprocessing to Episodic Context Reconstruction. Inspired by biological engrams, E-mem employs a heterogeneous hierarchical architecture where multiple assistant agents maintain uncompressed memory contexts, while a central master agent orchestrates global planning. Unlike passive retrieval, our mechanism empowers assistants to locally reason within activated segments, extracting context-aware evidence before aggregation. Evaluations on the LoCoMo benchmark demonstrate that E-mem achieves over 54\% F1, surpassing the state-of-the-art GAM by 7.75\%, while reducing token cost by over 70\%.
Paper Structure (22 sections, 9 equations, 4 figures, 7 tables, 1 algorithm)

This paper contains 22 sections, 9 equations, 4 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Architecture
  5. Memory Building and Storing
  6. Memory Activation via Multi-Pathway Routing
  7. Synergistic Reasoning and Response Generation
  8. Optimization: Latent State Caching
  9. Experiments
  10. Experiment Settings
  11. Performance Analysis
  12. Robustness Against Hallucination
  13. Ablation Study
  14. Cost Effectiveness Analysis
  15. Conclusion
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Traditional Memory System vs. E-mem
  • Figure 2: Overview of E-mem
  • Figure 3: Ablation Studies on Memory Routing
  • Figure 4: E-mem's state restoration enables successful answering, whereas traditional database-centric vector retrieval methods fail.