Accelerating Adaptive Retrieval Augmented Generation via Instruction-Driven Representation Reduction of Retrieval Overlaps

TL;DR

Adaptive-RAG improves answer quality via iterative retrieval but suffers efficiency costs due to overlapping content across rounds. The authors present IDR2, a model-agnostic framework consisting of Cross-Iterative Cache Sharing (CICS), Instruction-driven Deduplication Guidance Reinforcement (IDGR), and Information-Guided Parallel Generation (IGPG) to reduce redundant representations and enable parallel generation. Across multiple datasets and models, IDR2 achieves up to 2.0x end-to-end acceleration, with prefilling and decoding speedups of approximately 2.79x and 2.33x respectively, while preserving generation quality. The approach offers robust gains across different retrievers and scales, enhancing the practicality of Adaptive-RAG in real-world settings.

Abstract

Retrieval-augmented generation (RAG) has emerged as a pivotal method for expanding the knowledge of large language models. To handle complex queries more effectively, researchers developed Adaptive-RAG (A-RAG) to enhance the generated quality through multiple interactions with external knowledge bases. Despite its effectiveness, A-RAG exacerbates the pre-existing efficiency challenges inherent in RAG, which are attributable to its reliance on multiple iterations of generation. Existing A-RAG approaches process all retrieved contents from scratch. However, they ignore the situation where there is a significant overlap in the content of the retrieval results across rounds. The overlapping content is redundantly represented, which leads to a large proportion of repeated computations, thus affecting the overall efficiency. To address this issue, this paper introduces a model-agnostic approach that can be generally applied to A-RAG methods, which is dedicated to reducing the redundant representation process caused by the overlapping of retrieval results. Specifically, we use cache access and parallel generation to speed up the prefilling and decoding stages respectively. Additionally, we also propose an instruction-driven module to further guide the model to more effectively attend to each part of the content in a more suitable way for LLMs. Experiments show that our approach achieves 2.79 and 2.33 times significant acceleration on average for prefilling and decoding respectively while maintaining equal generation quality.

Figures (5)

• Figure 1: (a) The pipeline of A-RAG. (b) Analysis of document overlap between first and later retrievals (rounds 2-3) using 1000 2WikiMultihopQA samples.
• Figure 2: The pipeline of our IDR$_2$. the same color indicates the same document and representation.
• Figure 3: The x-axis represents the length of consecutive token combinations in the generated results. The y-axis represents the proportion of all combinations in the generated results that appear in the retrieved results. (LLaMA2-7B, 2WikiMultihopQA).
• Figure 4: The analysis of speedup for different numbers of retrieved documents.
• Figure 5: The detailed process of A-RAG based on a specific example , with LLaMA-13B.