Less is More for RAG: Information Gain Pruning for Generator-Aligned Reranking and Evidence Selection

TL;DR

This work tackles the budgeted retrieval-augmented generation problem where evidence must be admitted under a fixed context limit. It introduces Information Gain Pruning (IGP), a generator-aligned, deployment-friendly reranker that uses a Top-K normalized uncertainty proxy to compute Information Gain (IG) and prune low-utility passages before truncation. Across five QA benchmarks, multiple retrievers, and generator families, IGP consistently improves the end-to-end quality-cost frontier, particularly under multi-evidence budgets, by reducing final-stage token input by up to ~79% while boosting average F1 by ~12–20%. A key finding is the relevance–utility mismatch: offline relevance metrics (e.g., NDCG) weakly predict generation quality under budget constraints, underscoring the value of optimizing evidence utility rather than relevance alone. The approach is black-box, training-free, and broadly applicable, offering a practical knob Tp for admission control and robust gains across model families and scales, making budgeted RAG more reliable and cost-efficient in real deployments.

Abstract

Retrieval-augmented generation (RAG) grounds large language models with external evidence, but under a limited context budget, the key challenge is deciding which retrieved passages should be injected. We show that retrieval relevance metrics (e.g., NDCG) correlate weakly with end-to-end QA quality and can even become negatively correlated under multi-passage injection, where redundancy and mild conflicts destabilize generation. We propose \textbf{Information Gain Pruning (IGP)}, a deployment-friendly reranking-and-pruning module that selects evidence using a generator-aligned utility signal and filters weak or harmful passages before truncation, without changing existing budget interfaces. Across five open-domain QA benchmarks and multiple retrievers and generators, IGP consistently improves the quality--cost trade-off. In a representative multi-evidence setting, IGP delivers about +12--20% relative improvement in average F1 while reducing final-stage input tokens by roughly 76--79% compared to retriever-only baselines.

Figures (8)

• Figure 1: In RAG, better retrieval ranking does not necessarily yield better end-to-end generation. Under two evidence budgets (TopM=1 and TopM=5), the Spearman rank correlation between NDCG and end-to-end F1 is generally weak, and can be negative when multiple passages are injected (see \ref{['sec:relevance-vs-helpfulness']}).
• Figure 2: RAG pipeline comparison: relevance reranking vs. IGP. IGP replaces relevance scoring with an information-gain signal aligned with the generator, and can filter negative/weak-gain evidence; the budget-based truncation mechanism remains unchanged.
• Figure 3: Quality--cost Pareto frontiers under BM25 retrieval. We plot the cross-dataset average token-level F1 (higher is better) against the average final-stage input tokens TK (lower is better), averaging over NQ, TriviaQA, PopQA, SQuAD, and AmbigQA. Each curve is obtained by sweeping the evidence budget TopM; each point corresponds to a feasible deployment operating point under the same truncate budget interface. Curves closer to the upper-left indicate a better quality--cost trade-off.
• Figure 4: Sensitivity of IGP to the pruning threshold $T_p$ under TopM$=1$ (token-level F1).
• Figure 5: Sensitivity to TopK and maximum rollout length $MT$ (end-to-end F1). (a) A heatmap of F1 over a 2D grid of $(K,MT)$; (b) the marginal curve of TopK with fixed $MT$; (c) the marginal curve of $MT$ with fixed TopK. Here $K$ is the truncation size for TopK renormalization (Eq. \ref{['eq:topk_softmax_method']}), and $MT$ is the maximum rollout length with the effective-length rule in Eq. \ref{['eq:mt_trunc_rule_method']}. Overall, $MT$ plays a more dominant role, while TopK becomes robust once it reaches moderate values.