Structured Episodic Event Memory

TL;DR

This work tackles the long-standing challenge of maintaining coherent long-term memory in LLM-based agents by introducing Structured Episodic Event Memory (SEEM). SEEM combines a dynamic Episodic Memory Layer (EML) for narrative progression with a static Graph Memory Layer (GML) for relational facts, both provenance-grounded and integrated via a Reverse Provenance Expansion mechanism. The core innovations include Episodic Event Frames (EEFs) for event-centric representation and an associative fusion process to merge related events, along with a graph-based retrieval strategy that preserves structural dependencies. Empirical results on LoCoMo and LongMemEval show SEEM outperforms dense retrieval and prior memory-based systems, with strong gains in semantic trustworthiness and narrative coherence, and robust behavior under incremental construction. The work demonstrates practical gains for long-horizon reasoning in agents, while also outlining limitations related to computation, potential error propagation, and biases, pointing to future directions in efficiency and safety.

Abstract

Current approaches to memory in Large Language Models (LLMs) predominantly rely on static Retrieval-Augmented Generation (RAG), which often results in scattered retrieval and fails to capture the structural dependencies required for complex reasoning. For autonomous agents, these passive and flat architectures lack the cognitive organization necessary to model the dynamic and associative nature of long-term interaction. To address this, we propose Structured Episodic Event Memory (SEEM), a hierarchical framework that synergizes a graph memory layer for relational facts with a dynamic episodic memory layer for narrative progression. Grounded in cognitive frame theory, SEEM transforms interaction streams into structured Episodic Event Frames (EEFs) anchored by precise provenance pointers. Furthermore, we introduce an agentic associative fusion and Reverse Provenance Expansion (RPE) mechanism to reconstruct coherent narrative contexts from fragmented evidence. Experimental results on the LoCoMo and LongMemEval benchmarks demonstrate that SEEM significantly outperforms baselines, enabling agents to maintain superior narrative coherence and logical consistency.

Figures (7)

• Figure 1: Overview of the SEEM hierarchical memory architecture. The system transforms unstructured interaction passages into a dual-layer representation, integrating a semantic Graph Memory Layer for static facts with a structured Episodic Memory Layer for event-centric details. This hierarchical design enables the agent to effectively synergize stable factual knowledge with dynamic narrative contexts for coherent long-term reasoning.
• Figure 2: Overview of the associative consolidation and fusion. The $\mathcal{F}_{\text{ext}}$ first transforms raw interaction passages into structured EEFs, which are then processed by $\mathcal{F}_{\text{judge}}$ for the dynamic fusion of semantically related events. This mechanism aligns with associative consolidation to maintain a coherent and synthesized episodic memory store.
• Figure 3: Impact of the initial retrieval size ($|\mathcal{P}_{ret}|$).
• Figure 4: An illustrative example of a consolidated Episodic Event Frame (EEF) in the SEEM framework. This structured representation demonstrates how the associative fusion mechanism synthesizes multi-turn interactions into coherent, attribute-rich episodic units.
• Figure 5: The structured prompt for Episodic Event Frame Extraction ($\mathcal{F}_{ext}$). This initial stage of the SEEM pipeline converts unstructured interaction logs into discrete, attribute-rich event units, providing the grounded anchors necessary for long-term temporal and multi-hop reasoning.