Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Less LLM, More Documents: Searching for Improved RAG

Jingjie Ning, Yibo Kong, Yunfan Long, Jamie Callan

TL;DR

This work investigates whether enlarging the retrieval corpus can substitute for scaling the generator in retrieval-augmented generation (RAG) for open-domain QA. It introduces a principled, full-factorial framework that jointly varies corpus size and LLM scale, using ClueWeb22-A shards and Qwen3 models to evaluate on natural questions, TriviaQA, and WebQ. The key finding is that larger corpora consistently improve RAG performance, enabling smaller or mid-sized models to rival or surpass much larger models, with gains driven primarily by increased gold-answer coverage rather than changes in context utilization. Practically, the results advocate for deploying larger external corpora as an effective, deployable alternative to expanding LLM size, especially when inference budgets favor smaller models; the study also provides diagnostics like Gold Answer Coverage Rate and Utilization Ratio to guide budgeting between retriever and generator components.

Abstract

Retrieval-Augmented Generation (RAG) couples document retrieval with large language models (LLMs). While scaling generators improves accuracy, it also raises cost and limits deployability. We explore an orthogonal axis: enlarging the retriever's corpus to reduce reliance on large LLMs. Experimental results show that corpus scaling consistently strengthens RAG and can often serve as a substitute for increasing model size, though with diminishing returns at larger scales. Small- and mid-sized generators paired with larger corpora often rival much larger models with smaller corpora; mid-sized models tend to gain the most, while tiny and large models benefit less. Our analysis shows that improvements arise primarily from increased coverage of answer-bearing passages, while utilization efficiency remains largely unchanged. These findings establish a principled corpus-generator trade-off: investing in larger corpora offers an effective path to stronger RAG, often comparable to enlarging the LLM itself.

Less LLM, More Documents: Searching for Improved RAG

TL;DR

This work investigates whether enlarging the retrieval corpus can substitute for scaling the generator in retrieval-augmented generation (RAG) for open-domain QA. It introduces a principled, full-factorial framework that jointly varies corpus size and LLM scale, using ClueWeb22-A shards and Qwen3 models to evaluate on natural questions, TriviaQA, and WebQ. The key finding is that larger corpora consistently improve RAG performance, enabling smaller or mid-sized models to rival or surpass much larger models, with gains driven primarily by increased gold-answer coverage rather than changes in context utilization. Practically, the results advocate for deploying larger external corpora as an effective, deployable alternative to expanding LLM size, especially when inference budgets favor smaller models; the study also provides diagnostics like Gold Answer Coverage Rate and Utilization Ratio to guide budgeting between retriever and generator components.

Abstract

Retrieval-Augmented Generation (RAG) couples document retrieval with large language models (LLMs). While scaling generators improves accuracy, it also raises cost and limits deployability. We explore an orthogonal axis: enlarging the retriever's corpus to reduce reliance on large LLMs. Experimental results show that corpus scaling consistently strengthens RAG and can often serve as a substitute for increasing model size, though with diminishing returns at larger scales. Small- and mid-sized generators paired with larger corpora often rival much larger models with smaller corpora; mid-sized models tend to gain the most, while tiny and large models benefit less. Our analysis shows that improvements arise primarily from increased coverage of answer-bearing passages, while utilization efficiency remains largely unchanged. These findings establish a principled corpus-generator trade-off: investing in larger corpora offers an effective path to stronger RAG, often comparable to enlarging the LLM itself.

Paper Structure

This paper contains 24 sections, 5 equations, 10 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Retriever: Corpus Scaling
  5. Generator
  6. Trade-off Formalization
  7. Experiment
  8. Benchmarks
  9. Evaluation Metrics
  10. Retriever: Implementation Details
  11. Generator: Implementation Details
  12. Results & Analysis
  13. The Effect of Corpus Scaling as Compensation
  14. Compensation Effect.
  15. Corpus Quality vs. Quantity.
  16. ...and 9 more sections

Figures (10)

  • Figure 1: F1 Gains from Scaling on NQ
  • Figure 2: EM Gains from Scaling on NQ
  • Figure 3: F1 and Catch-up Thresholds under Reversed Corpus Scaling. Left: F1 when using forward (FWD) vs. reversed (REV) corpus scaling order. Right: corresponding catch-up thresholds.
  • Figure 4: Gold Answer Coverage Rate for Forward Scaling
  • Figure 5: Known Rate on NQ
  • ...and 5 more figures