DH-RAG: A Dynamic Historical Context-Powered Retrieval-Augmented Generation Method for Multi-Turn Dialogue

TL;DR

DH-RAG tackles the limitation of static knowledge bases in retrieval-augmented generation for multi-turn dialogue by introducing a Dynamic Historical Context framework. It comprises a History-Learning Based Query Reconstruction Module, a Dynamic History Information Updating Module, and a Dynamic Historical Information Database, augmented by Historical Query Clustering, Hierarchical Matching, and Chain of Thought Tracking. The method reconstructs queries using both static knowledge and short-term history, and continuously updates the historical database to reflect evolving conversations, enabling coherent and contextually grounded responses. Empirical results across MobileCS2, modified TriviaQA/PopQA, CoQA, and TopiOCQA demonstrate that DH-RAG outperforms baselines in BLEU and F1, with only modest runtime overhead, highlighting its practical potential for dynamic, memory-augmented dialogue systems. Overall, the work advances RAG by modeling memory dynamics and offering scalable mechanisms to leverage evolving conversational context.

Abstract

Retrieval-Augmented Generation (RAG) systems have shown substantial benefits in applications such as question answering and multi-turn dialogue \citep{lewis2020retrieval}. However, traditional RAG methods, while leveraging static knowledge bases, often overlook the potential of dynamic historical information in ongoing conversations. To bridge this gap, we introduce DH-RAG, a Dynamic Historical Context-Powered Retrieval-Augmented Generation Method for Multi-Turn Dialogue. DH-RAG is inspired by human cognitive processes that utilize both long-term memory and immediate historical context in conversational responses \citep{stafford1987conversational}. DH-RAG is structured around two principal components: a History-Learning based Query Reconstruction Module, designed to generate effective queries by synthesizing current and prior interactions, and a Dynamic History Information Updating Module, which continually refreshes historical context throughout the dialogue. The center of DH-RAG is a Dynamic Historical Information database, which is further refined by three strategies within the Query Reconstruction Module: Historical Query Clustering, Hierarchical Matching, and Chain of Thought Tracking. Experimental evaluations show that DH-RAG significantly surpasses conventional models on several benchmarks, enhancing response relevance, coherence, and dialogue quality.

Figures (6)

• Figure 1: Difference between existing RAG methods and our DH-RAG method.
• Figure 2: Overall pipeline of our DH-RAG method.
• Figure 3: Illustration of two key strategies in DH-RAG.
• Figure 4: Distribution of query clusters showing the semantic categorization across different types of customer service interactions. Cluster $C_2$ (Business Operations) and $C_9$ (Package Recommendation) demonstrate the highest frequencies, indicating prevalent customer inquiry patterns.
• Figure 5: Distribution of reasoning chain lengths in DH-RAG's Chain of Thought tracking process, showing the majority of reasoning chains contain 2-3 steps (150 instances), with decreasing frequency for longer chains.