Recursive Abstractive Processing for Retrieval in Dynamic Datasets

TL;DR

This work proposes a new algorithm to efficiently maintain the recursive-abstractive tree structure in dynamic datasets, without compromising performance, and introduces a novel post-retrieval method that applies query-focused recursive abstractive processing to substantially improve context quality.

Abstract

Recent retrieval-augmented models enhance basic methods by building a hierarchical structure over retrieved text chunks through recursive embedding, clustering, and summarization. The most relevant information is then retrieved from both the original text and generated summaries. However, such approaches face limitations with dynamic datasets, where adding or removing documents over time complicates the updating of hierarchical representations formed through clustering. We propose a new algorithm to efficiently maintain the recursive-abstractive tree structure in dynamic datasets, without compromising performance. Additionally, we introduce a novel post-retrieval method that applies query-focused recursive abstractive processing to substantially improve context quality. Our method overcomes the limitations of other approaches by functioning as a black-box post-retrieval layer compatible with any retrieval algorithm. Both algorithms are validated through extensive experiments on real-world datasets, demonstrating their effectiveness in handling dynamic data and improving retrieval performance.

Figures (18)

• Figure 1: Retrieval pipeline with postQFRAP: we first retrieve from a dataset $k_0$ chunks relevant to the query, then we build a query-focused recursive-abstractive tree on those chunks. Finally, we summarize the contents of the root layer of that tree to get the context that is passed to the LLM.
• Figure 2: Evaluation of adRAP (Section \ref{['sec:adRAP']}) on 3 datasets.
• Figure 3: Percentage of Wins, Ties and Losses for adRAP vs other algorithms on MultiHop.
• Figure 4: Percentage of Wins, Ties and Losses for adRAP vs other algorithms on QASPER.
• Figure 5: Percentage of Wins, Ties and Losses for adRAP vs other algorithms on QuALITY.