Enhancing Long-Term Memory using Hierarchical Aggregate Tree for Retrieval Augmented Generation

TL;DR

The paper addresses the challenge of long-term dialogue reasoning under restricted context windows by introducing Hierarchical Aggregate Tree (HAT) memory and a GPT-based Memory Agent that performs conditional traversal to retrieve compact, relevant context. By formulating memory retrieval as an optimization over tree traversal, the approach achieves more coherent and grounded multi-turn responses without exponential parameter growth. Empirical results show that GPTAgent-based traversal outperforms naive baselines like BFS/DFS and context- and memory-based comparators on standard metrics, while memory fidelity remains high compared to gold references. The proposed framework offers a scalable path to memory-augmented retrieval for long-form conversations, with potential extensions to hybrid representations and more efficient search strategies.

Abstract

Large language models have limited context capacity, hindering reasoning over long conversations. We propose the Hierarchical Aggregate Tree memory structure to recursively aggregate relevant dialogue context through conditional tree traversals. HAT encapsulates information from children nodes, enabling broad coverage with depth control. We formulate finding best context as optimal tree traversal. Experiments show HAT improves dialog coherence and summary quality over baseline contexts, demonstrating the techniques effectiveness for multi turn reasoning without exponential parameter growth. This memory augmentation enables more consistent, grounded longform conversations from LLMs

Figures (2)

• Figure 1: Overview of our approach. Given a user query, the memory module is responsible to give a relevant context by traversing the HAT. The LLM then generates response for the user query, given the context.
• Figure 2: Illustration of HAT, with example aggregation function as simple concatenation and memory length of 2.