Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Preference-Aware Memory Update for Long-Term LLM Agents

Haoran Sun, Zekun Zhang, Shaoning Zeng

TL;DR

The paper tackles the problem of memory updating in long-term LLM agents by introducing PAMU, a modular mechanism that fuses short-term (sliding window) and long-term (EMA) preferences to create a dynamic, preference-aware memory. It combines a Preference Extractor, a Change Perception Module, and prompt-level control to adapt generation without fine-tuning, supported by a Bayesian/Kalman-filter-inspired rationale and a change-detection trigger. Evaluations on the LoCoMo dataset show PAMU improves memory-guided generation across multiple baselines, with ablations confirming the necessity of each component. The approach offers interpretable, real-time personalization in long-term dialogues, advancing practical deployment of memory-augmented LLM agents.

Abstract

One of the key factors influencing the reasoning capabilities of LLM-based agents is their ability to leverage long-term memory. Integrating long-term memory mechanisms allows agents to make informed decisions grounded in historical interactions. While recent advances have significantly improved the storage and retrieval components, by encoding memory into dense vectors for similarity search or organizing memory as structured knowledge graphs most existing approaches fall short in memory updating. In particular, they lack mechanisms for dynamically refining preference memory representations in response to evolving user behaviors and contexts. To address this gap, we propose a Preference-Aware Memory Update Mechanism (PAMU) that enables dynamic and personalized memory refinement. By integrating sliding window averages (SW) with exponential moving averages (EMA), PAMU constructs a fused preference-aware representation that captures both short-term fluctuations and long-term user tendencies. We conduct experiments on five task scenarios of the LoCoMo dataset, and the results show that our mechanism can significantly improve the output quality of LLM in five baselines, validating its effectiveness in long-term conversations.

Preference-Aware Memory Update for Long-Term LLM Agents

TL;DR

The paper tackles the problem of memory updating in long-term LLM agents by introducing PAMU, a modular mechanism that fuses short-term (sliding window) and long-term (EMA) preferences to create a dynamic, preference-aware memory. It combines a Preference Extractor, a Change Perception Module, and prompt-level control to adapt generation without fine-tuning, supported by a Bayesian/Kalman-filter-inspired rationale and a change-detection trigger. Evaluations on the LoCoMo dataset show PAMU improves memory-guided generation across multiple baselines, with ablations confirming the necessity of each component. The approach offers interpretable, real-time personalization in long-term dialogues, advancing practical deployment of memory-augmented LLM agents.

Abstract

One of the key factors influencing the reasoning capabilities of LLM-based agents is their ability to leverage long-term memory. Integrating long-term memory mechanisms allows agents to make informed decisions grounded in historical interactions. While recent advances have significantly improved the storage and retrieval components, by encoding memory into dense vectors for similarity search or organizing memory as structured knowledge graphs most existing approaches fall short in memory updating. In particular, they lack mechanisms for dynamically refining preference memory representations in response to evolving user behaviors and contexts. To address this gap, we propose a Preference-Aware Memory Update Mechanism (PAMU) that enables dynamic and personalized memory refinement. By integrating sliding window averages (SW) with exponential moving averages (EMA), PAMU constructs a fused preference-aware representation that captures both short-term fluctuations and long-term user tendencies. We conduct experiments on five task scenarios of the LoCoMo dataset, and the results show that our mechanism can significantly improve the output quality of LLM in five baselines, validating its effectiveness in long-term conversations.

Paper Structure

This paper contains 22 sections, 16 equations, 1 figure, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Preference Extractor
  5. Preference Change Perception Mechanism
  6. Dynamic Modeling of Continuous Preference Dimensions.
  7. Dynamic Modeling of Categorical Preference Dimensions.
  8. Preference-Guided Prompting
  9. Motivation and Basis
  10. 1. Bayesian Estimation View: Probabilistic Optimality.
  11. 2. Kalman Filtering Approximation: Recursive Preference Tracking.
  12. 3. Change Detection Signal: Behavioral Adaptation Trigger.
  13. Experiment
  14. Setup
  15. Dataset and Evaluation Metrics.
  16. ...and 7 more sections

Figures (1)

  • Figure 1: Illustration of PAMU method. PAMU extracts user preferences from dialogue, models short- and long-term trends via SW and EMA, detects preference shifts, and updates the prompt to guide personalized generation.