Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agents

Miao Su, Yucan Guo, Zhongni Hou, Long Bai, Zixuan Li, Yufei Zhang, Guojun Yin, Wei Lin, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng

TL;DR

The paper addresses the misalignment between when memories are stored and when events occur, as well as fragmentation of long-term memories in LLM agents. It proposes Temporal Semantic Memory (TSM), which builds a semantic timeline via a temporal knowledge graph and consolidates episodic data into durative memories. During retrieval, TSM applies semantic-time constraints to access time-valid, duration-consistent context, boosting temporal reasoning and multi-session understanding. On LongMemEval and LoCoMo, TSM outperforms strong baselines across multiple backbones, demonstrating the value of semantic-time grounding for personalized, long-horizon AI assistants.

Abstract

Memory enables Large Language Model (LLM) agents to perceive, store, and use information from past dialogues, which is essential for personalization. However, existing methods fail to properly model the temporal dimension of memory in two aspects: 1) Temporal inaccuracy: memories are organized by dialogue time rather than their actual occurrence time; 2) Temporal fragmentation: existing methods focus on point-wise memory, losing durative information that captures persistent states and evolving patterns. To address these limitations, we propose Temporal Semantic Memory (TSM), a memory framework that models semantic time for point-wise memory and supports the construction and utilization of durative memory. During memory construction, it first builds a semantic timeline rather than a dialogue one. Then, it consolidates temporally continuous and semantically related information into a durative memory. During memory utilization, it incorporates the query's temporal intent on the semantic timeline, enabling the retrieval of temporally appropriate durative memories and providing time-valid, duration-consistent context to support response generation. Experiments on LongMemEval and LoCoMo show that TSM consistently outperforms existing methods and achieves up to 12.2% absolute improvement in accuracy, demonstrating the effectiveness of the proposed method.

Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agents

TL;DR

The paper addresses the misalignment between when memories are stored and when events occur, as well as fragmentation of long-term memories in LLM agents. It proposes Temporal Semantic Memory (TSM), which builds a semantic timeline via a temporal knowledge graph and consolidates episodic data into durative memories. During retrieval, TSM applies semantic-time constraints to access time-valid, duration-consistent context, boosting temporal reasoning and multi-session understanding. On LongMemEval and LoCoMo, TSM outperforms strong baselines across multiple backbones, demonstrating the value of semantic-time grounding for personalized, long-horizon AI assistants.

Abstract

Memory enables Large Language Model (LLM) agents to perceive, store, and use information from past dialogues, which is essential for personalization. However, existing methods fail to properly model the temporal dimension of memory in two aspects: 1) Temporal inaccuracy: memories are organized by dialogue time rather than their actual occurrence time; 2) Temporal fragmentation: existing methods focus on point-wise memory, losing durative information that captures persistent states and evolving patterns. To address these limitations, we propose Temporal Semantic Memory (TSM), a memory framework that models semantic time for point-wise memory and supports the construction and utilization of durative memory. During memory construction, it first builds a semantic timeline rather than a dialogue one. Then, it consolidates temporally continuous and semantically related information into a durative memory. During memory utilization, it incorporates the query's temporal intent on the semantic timeline, enabling the retrieval of temporally appropriate durative memories and providing time-valid, duration-consistent context to support response generation. Experiments on LongMemEval and LoCoMo show that TSM consistently outperforms existing methods and achieves up to 12.2% absolute improvement in accuracy, demonstrating the effectiveness of the proposed method.
Paper Structure (27 sections, 20 equations, 3 figures, 4 tables)

This paper contains 27 sections, 20 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Agent Memory
  4. Graph-structured Memory
  5. Methodology
  6. Preliminary
  7. Overview
  8. Duration-aware Memory Construction
  9. Episodic Memory Construction
  10. Durative Memory Construction
  11. Semantic-time Guided Memory Utilization
  12. Hierarchical Memory Update
  13. Experiments
  14. Experimental Setup
  15. Main Results
  16. ...and 12 more sections

Figures (3)

  • Figure 1: Comparison of existing methods vs. TSM with semantic timeline and durative memories.
  • Figure 2: The overall framework of TSM. Memory consolidation constructs a temporal knowledge graph from episodic memory and subsequently consolidates it into time-aware durative memory. Memory utilization retrieves accurate memories by applying semantic-temporal constraints.
  • Figure 3: Case Study of TSM.