Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RAGBoost: Efficient Retrieval-Augmented Generation with Accuracy-Preserving Context Reuse

Yinsicheng Jiang, Yeqi Huang, Liang Cheng, Cheng Deng, Xuan Sun, Luo Mai

TL;DR

RAGBoost tackles the prefill latency of retrieval-augmented generation under long-context demands by enabling accuracy-preserving context reuse across multi-turn and multi-session workloads. It introduces a Context Index, Context Ordering, Context Deduplication, and lightweight contextual hints to maximize KV-cache reuse without sacrificing reasoning quality. The approach delivers 1.5–3× improvements in prefill throughput and up to 4.1% accuracy gains across diverse RAG datasets, with additional gains in agentic AI tasks; it remains compatible with standard retrievers and LLM engines and is open-sourced to spur broader context-engineering research. This work demonstrates that strategic organization and reuse of retrieved content can unlock efficient long-context RAG at scale, offering practical benefits for online services and offline data-synthesis pipelines alike.

Abstract

Retrieval-augmented generation (RAG) enhances large language models (LLMs) with retrieved context but often suffers from downgraded prefill performance as modern applications demand longer and more complex inputs. Existing caching techniques either preserve accuracy with low cache reuse or improve reuse at the cost of degraded reasoning quality. We present RAGBoost, an efficient RAG system that achieves high cache reuse without sacrificing accuracy through accuracy-preserving context reuse. RAGBoost detects overlapping retrieved items across concurrent sessions and multi-turn interactions, using efficient context indexing, ordering, and de-duplication to maximize reuse, while lightweight contextual hints maintain reasoning fidelity. It integrates seamlessly with existing LLM inference engines and improves their prefill performance by 1.5-3X over state-of-the-art methods, while preserving or even enhancing reasoning accuracy across diverse RAG and agentic AI workloads. Our code is released at: https://github.com/Edinburgh-AgenticAI/RAGBoost.

RAGBoost: Efficient Retrieval-Augmented Generation with Accuracy-Preserving Context Reuse

TL;DR

RAGBoost tackles the prefill latency of retrieval-augmented generation under long-context demands by enabling accuracy-preserving context reuse across multi-turn and multi-session workloads. It introduces a Context Index, Context Ordering, Context Deduplication, and lightweight contextual hints to maximize KV-cache reuse without sacrificing reasoning quality. The approach delivers 1.5–3× improvements in prefill throughput and up to 4.1% accuracy gains across diverse RAG datasets, with additional gains in agentic AI tasks; it remains compatible with standard retrievers and LLM engines and is open-sourced to spur broader context-engineering research. This work demonstrates that strategic organization and reuse of retrieved content can unlock efficient long-context RAG at scale, offering practical benefits for online services and offline data-synthesis pipelines alike.

Abstract

Retrieval-augmented generation (RAG) enhances large language models (LLMs) with retrieved context but often suffers from downgraded prefill performance as modern applications demand longer and more complex inputs. Existing caching techniques either preserve accuracy with low cache reuse or improve reuse at the cost of degraded reasoning quality. We present RAGBoost, an efficient RAG system that achieves high cache reuse without sacrificing accuracy through accuracy-preserving context reuse. RAGBoost detects overlapping retrieved items across concurrent sessions and multi-turn interactions, using efficient context indexing, ordering, and de-duplication to maximize reuse, while lightweight contextual hints maintain reasoning fidelity. It integrates seamlessly with existing LLM inference engines and improves their prefill performance by 1.5-3X over state-of-the-art methods, while preserving or even enhancing reasoning accuracy across diverse RAG and agentic AI workloads. Our code is released at: https://github.com/Edinburgh-AgenticAI/RAGBoost.

Paper Structure

This paper contains 23 sections, 1 equation, 10 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Retrieval-augmented generation systems
  4. Emerging challenge: longer context retrieval
  5. Issues of existing KV cache reuse methods
  6. Design Overview
  7. Observation: significant overlap in retrieval
  8. Design opportunities for context reuse
  9. RAGBoost system overview
  10. Context Index
  11. Key designs for context index
  12. Key operations with context index
  13. Context Ordering
  14. Context ordering algorithm
  15. Scheduling requests with ordered contexts
  16. ...and 8 more sections

Figures (10)

  • Figure 1: Overview of a RAG system.
  • Figure 2: Context overlap and context reuse opportunities in RAG.
  • Figure 3: System Overview of RAGBoost.
  • Figure 4: Context index construction with prefix-cache semantics.
  • Figure 5: Context ordering: (1) find best-match nodes; (2) reorder by shared prefix and append as child; (3) output the ordered context.
  • ...and 5 more figures