Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ES-Mem: Event Segmentation-Based Memory for Long-Term Dialogue Agents

Huhai Zou, Tianhao Sun, Chuanjiang He, Yu Tian, Zhenyang Li, Li Jin, Nayu Liu, Jiang Zhong, Kaiwen Wei

TL;DR

ES-Mem tackles the problem of memory fragmentation and structure-agnostic retrieval in long-term dialogue agents. It introduces a dynamic event segmentation module, grounded in Topic Coherence and Intent transitions, and a layered memory architecture with refined boundaries, event summaries, and raw context, all integrated via a coarse-to-fine retrieval pipeline anchored by event boundaries. Empirical results on LoCoMo and LongMemEval-S show ES-Mem consistently outperforms strong baselines, with the event segmentation module also transferring effectively to dialogue segmentation benchmarks. The approach improves context localization, preserves discourse semantics, and offers a scalable memory solution for long-running conversations with potential extensions to multimodal data and memory dynamics.

Abstract

Memory is critical for dialogue agents to maintain coherence and enable continuous adaptation in long-term interactions. While existing memory mechanisms offer basic storage and retrieval capabilities, they are hindered by two primary limitations: (1) rigid memory granularity often disrupts semantic integrity, resulting in fragmented and incoherent memory units; (2) prevalent flat retrieval paradigms rely solely on surface-level semantic similarity, neglecting the structural cues of discourse required to navigate and locate specific episodic contexts. To mitigate these limitations, drawing inspiration from Event Segmentation Theory, we propose ES-Mem, a framework incorporating two core components: (1) a dynamic event segmentation module that partitions long-term interactions into semantically coherent events with distinct boundaries; (2) a hierarchical memory architecture that constructs multi-layered memories and leverages boundary semantics to anchor specific episodic memory for precise context localization. Evaluations on two memory benchmarks demonstrate that ES-Mem yields consistent performance gains over baseline methods. Furthermore, the proposed event segmentation module exhibits robust applicability on dialogue segmentation datasets.

ES-Mem: Event Segmentation-Based Memory for Long-Term Dialogue Agents

TL;DR

ES-Mem tackles the problem of memory fragmentation and structure-agnostic retrieval in long-term dialogue agents. It introduces a dynamic event segmentation module, grounded in Topic Coherence and Intent transitions, and a layered memory architecture with refined boundaries, event summaries, and raw context, all integrated via a coarse-to-fine retrieval pipeline anchored by event boundaries. Empirical results on LoCoMo and LongMemEval-S show ES-Mem consistently outperforms strong baselines, with the event segmentation module also transferring effectively to dialogue segmentation benchmarks. The approach improves context localization, preserves discourse semantics, and offers a scalable memory solution for long-running conversations with potential extensions to multimodal data and memory dynamics.

Abstract

Memory is critical for dialogue agents to maintain coherence and enable continuous adaptation in long-term interactions. While existing memory mechanisms offer basic storage and retrieval capabilities, they are hindered by two primary limitations: (1) rigid memory granularity often disrupts semantic integrity, resulting in fragmented and incoherent memory units; (2) prevalent flat retrieval paradigms rely solely on surface-level semantic similarity, neglecting the structural cues of discourse required to navigate and locate specific episodic contexts. To mitigate these limitations, drawing inspiration from Event Segmentation Theory, we propose ES-Mem, a framework incorporating two core components: (1) a dynamic event segmentation module that partitions long-term interactions into semantically coherent events with distinct boundaries; (2) a hierarchical memory architecture that constructs multi-layered memories and leverages boundary semantics to anchor specific episodic memory for precise context localization. Evaluations on two memory benchmarks demonstrate that ES-Mem yields consistent performance gains over baseline methods. Furthermore, the proposed event segmentation module exhibits robust applicability on dialogue segmentation datasets.
Paper Structure (45 sections, 11 equations, 8 figures, 4 tables)

This paper contains 45 sections, 11 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Memory for Dialogue Agents
  4. Dynamic Memory Granularity
  5. Methodology
  6. Dynamic Event Segmentation
  7. Stage 1: Topic Coherence Detection.
  8. Stage 2: Intent-Aware Boundary Refinement.
  9. Layered Memory Construction
  10. Level 1: Refined Event Boundaries ($b_i^{\text{ref}}$).
  11. Level 2: Event Summaries ($s_i$).
  12. Level 3: Raw Context ($r_i$).
  13. Coarse-to-Fine Retrieval
  14. Stage 1: Boundary Scanning (Anchor Retrieval).
  15. Stage 2: Activated Memory Interval Expansion.
  16. ...and 30 more sections

Figures (8)

  • Figure 1: Comparison between existing memory methods and ES-Mem. Existing methods with fixed granularity often sever semantic dependencies, causing flat retrieval to recall inaccurate contexts. Conversely, ES-Mem integrates Event Segmentation Theory (EST) to dynamically structure memory. By using boundary anchors, it precisely locates episodic contexts.
  • Figure 2: Overview of ES-Mem. The framework consists of three modules: Dynamic Event Segmentation partitions the dialogue stream into semantic events; Layered Memory Construction builds multi-level storage with boundary anchors; and Coarse-to-Fine Retrieval utilizes these anchors to precisely locate and expand relevant contexts.
  • Figure 3: Results of ablation experiments on LoCoMo benchmark. We use GPT-4o-mini as the backbone LLM.
  • Figure 4: Case study in long-term dialogue scenarios.
  • Figure 5: Results of memory retrieval parameter $K$. Using GPT-4o-mini as the base LLM model.
  • ...and 3 more figures