Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ever-Evolving Memory by Blending and Refining the Past

Seo Hyun Kim, Keummin Ka, Yohan Jo, Seung-won Hwang, Dongha Lee, Jinyoung Yeo

TL;DR

The paper addresses the challenge of endowing chatbots with robust, long-term memory. It introduces CREEM, a Contextualized Refinement based Ever-Evolving Memory framework that blends past memories with current dialogue and refines the memory stream to forget outdated or conflicting information, treating memory construction and response generation as intertwined. CREEM employs contextual search to broaden memory retrieval, a blending step to generate new insights from past and present information, and a refining step to maintain consistency, with insights infused into response generation. Extensive experiments on MSC and CC datasets demonstrate that CREEM achieves superior memory quality across integration, consistency, and sophistication, and yields more coherent, memory-informed responses compared to strong baselines, without requiring extensive fine-tuning. The work highlights the practical impact of memory-informed dialogue and provides a foundation for future improvements in memory retrievers and larger-scale long-term conversational datasets.

Abstract

For a human-like chatbot, constructing a long-term memory is crucial. However, current large language models often lack this capability, leading to instances of missing important user information or redundantly asking for the same information, thereby diminishing conversation quality. To effectively construct memory, it is crucial to seamlessly connect past and present information, while also possessing the ability to forget obstructive information. To address these challenges, we propose CREEM, a novel memory system for long-term conversation. Improving upon existing approaches that construct memory based solely on current sessions, CREEM blends past memories during memory formation. Additionally, we introduce a refining process to handle redundant or outdated information. Unlike traditional paradigms, we view responding and memory construction as inseparable tasks. The blending process, which creates new memories, also serves as a reasoning step for response generation by informing the connection between past and present. Through evaluation, we demonstrate that CREEM enhances both memory and response qualities in multi-session personalized dialogues.

Ever-Evolving Memory by Blending and Refining the Past

TL;DR

The paper addresses the challenge of endowing chatbots with robust, long-term memory. It introduces CREEM, a Contextualized Refinement based Ever-Evolving Memory framework that blends past memories with current dialogue and refines the memory stream to forget outdated or conflicting information, treating memory construction and response generation as intertwined. CREEM employs contextual search to broaden memory retrieval, a blending step to generate new insights from past and present information, and a refining step to maintain consistency, with insights infused into response generation. Extensive experiments on MSC and CC datasets demonstrate that CREEM achieves superior memory quality across integration, consistency, and sophistication, and yields more coherent, memory-informed responses compared to strong baselines, without requiring extensive fine-tuning. The work highlights the practical impact of memory-informed dialogue and provides a foundation for future improvements in memory retrievers and larger-scale long-term conversational datasets.

Abstract

For a human-like chatbot, constructing a long-term memory is crucial. However, current large language models often lack this capability, leading to instances of missing important user information or redundantly asking for the same information, thereby diminishing conversation quality. To effectively construct memory, it is crucial to seamlessly connect past and present information, while also possessing the ability to forget obstructive information. To address these challenges, we propose CREEM, a novel memory system for long-term conversation. Improving upon existing approaches that construct memory based solely on current sessions, CREEM blends past memories during memory formation. Additionally, we introduce a refining process to handle redundant or outdated information. Unlike traditional paradigms, we view responding and memory construction as inseparable tasks. The blending process, which creates new memories, also serves as a reasoning step for response generation by informing the connection between past and present. Through evaluation, we demonstrate that CREEM enhances both memory and response qualities in multi-session personalized dialogues.
Paper Structure (51 sections, 1 equation, 4 figures, 7 tables, 1 algorithm)

This paper contains 51 sections, 1 equation, 4 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Long-term Memory Systems
  4. Forgetting in Long-term Memory
  5. Proposed Approach: CREEM
  6. Contextual Search
  7. Blend and Refine
  8. Blending.
  9. Refining.
  10. Insight Infused Response Generation
  11. Experiments
  12. Datasets
  13. Models and Settings
  14. $\text{SumMem}_\text{MSC}$
  15. $\textsc{ReBoT}_\text{CC}$
  16. ...and 36 more sections

Figures (4)

  • Figure 1: Motivating examples where our memory (Blend + Refine) is needed compared to not refining (Case 1) and deleting all contradictories (Case 2).
  • Figure 2: Overall memory update process of CREEM.
  • Figure 3: Degree of Contradiction.
  • Figure 4: Example episode of how past memory is blended and utilized. Change in Speaker A’s hair color choice is reflected in the later sessions.