Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models

Qingyue Wang, Yanhe Fu, Yanan Cao, Shuai Wang, Zhiliang Tian, Liang Ding

TL;DR

The paper tackles the problem of memory loss in long-term dialogues by introducing a recursive memory mechanism that prompts a large language model to generate and continually update a compact memory representation across sessions. It decomposes the task into memory iteration and memory-based response generation, formalizing the interaction with a memory-driven factorization and implementing a memory-prompting schema that evolves with the dialogue. Across MSC and CareCall datasets, the approach yields improved coherence and consistency, demonstrating robustness across open and closed LLMs and offering complementary gains to retrieval-based and long-context methods. The method provides a practical, plug-in solution for extending long-range dialogue ability and modeling extremely long contexts, with code and prompts released to facilitate adoption and further research.

Abstract

Recently, large language models (LLMs), such as GPT-4, stand out remarkable conversational abilities, enabling them to engage in dynamic and contextually relevant dialogues across a wide range of topics. However, given a long conversation, these chatbots fail to recall past information and tend to generate inconsistent responses. To address this, we propose to recursively generate summaries/ memory using large language models (LLMs) to enhance long-term memory ability. Specifically, our method first stimulates LLMs to memorize small dialogue contexts and then recursively produce new memory using previous memory and following contexts. Finally, the chatbot can easily generate a highly consistent response with the help of the latest memory. We evaluate our method on both open and closed LLMs, and the experiments on the widely-used public dataset show that our method can generate more consistent responses in a long-context conversation. Also, we show that our strategy could nicely complement both long-context (e.g., 8K and 16K) and retrieval-enhanced LLMs, bringing further long-term dialogue performance. Notably, our method is a potential solution to enable the LLM to model the extremely long context. The code and scripts are released.

Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models

TL;DR

The paper tackles the problem of memory loss in long-term dialogues by introducing a recursive memory mechanism that prompts a large language model to generate and continually update a compact memory representation across sessions. It decomposes the task into memory iteration and memory-based response generation, formalizing the interaction with a memory-driven factorization and implementing a memory-prompting schema that evolves with the dialogue. Across MSC and CareCall datasets, the approach yields improved coherence and consistency, demonstrating robustness across open and closed LLMs and offering complementary gains to retrieval-based and long-context methods. The method provides a practical, plug-in solution for extending long-range dialogue ability and modeling extremely long contexts, with code and prompts released to facilitate adoption and further research.

Abstract

Recently, large language models (LLMs), such as GPT-4, stand out remarkable conversational abilities, enabling them to engage in dynamic and contextually relevant dialogues across a wide range of topics. However, given a long conversation, these chatbots fail to recall past information and tend to generate inconsistent responses. To address this, we propose to recursively generate summaries/ memory using large language models (LLMs) to enhance long-term memory ability. Specifically, our method first stimulates LLMs to memorize small dialogue contexts and then recursively produce new memory using previous memory and following contexts. Finally, the chatbot can easily generate a highly consistent response with the help of the latest memory. We evaluate our method on both open and closed LLMs, and the experiments on the widely-used public dataset show that our method can generate more consistent responses in a long-context conversation. Also, we show that our strategy could nicely complement both long-context (e.g., 8K and 16K) and retrieval-enhanced LLMs, bringing further long-term dialogue performance. Notably, our method is a potential solution to enable the LLM to model the extremely long context. The code and scripts are released.
Paper Structure (31 sections, 3 equations, 8 figures, 17 tables, 1 algorithm)

This paper contains 31 sections, 3 equations, 8 figures, 17 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Large Language Models
  4. Long-term Open-Domain Dialogue
  5. Approach Overview
  6. Approach
  7. Memory Iteration
  8. Prompt Construction
  9. Memory-based Response Generation
  10. Algorithm
  11. Experimental Settings
  12. Datasets
  13. MSC dataset
  14. Carecall
  15. Evaluation Metrics
  16. ...and 16 more sections

Figures (8)

  • Figure 1: A long-term conversation example from the Multi-Session Chat Dataset xu-etal-2022-beyond. When the user refers back to previous subjects (i.e., composing music), even the ChatGPT (gpt-turbo-3.5-0301 version) generates an inconsistent response.
  • Figure 2: Comparison among baselines and ours. The "U", "S", "T", and "M" are abbreviations for the Utterance, Session, dialog Topic, and Memory. The red dashed box refers to the memory used to generate the response.
  • Figure 3: The schematic overview of our method. The model uses the first session to generate initial memory (green arrows), then updates the memory when the second session ends (yellow arrows), and generates a response using the latest memory at the third session (blue arrows).
  • Figure 4: Comparative win rate of our method and competitive baselines, including ChatGPT, ChatGPT-MemoChat, and ChatGPT-MemoryBank.
  • Figure 5: The evaluation on generated memory on ChatGPT-MemoryBank and ours. The "P" and "R" refer the precision and recall, respectively.
  • ...and 3 more figures