AdaGReS:Adaptive Greedy Context Selection via Redundancy-Aware Scoring for Token-Budgeted RAG

TL;DR

AdaGReS addresses redundancy and token-budget inefficiency in retrieval-augmented generation by formulating a redundancy-aware set-level objective F(q,C)=α S_qC(q,C)−β S_CC(C) and deriving a closed-form adaptive β^* that adapts to candidate-pool statistics and the token budget. It establishes ε-approximate submodularity for the objective, providing near-optimality guarantees for greedy selection under practical embedding distributions, and demonstrates empirical improvements on open-domain Natural Questions and a high-redundancy biomedical corpus in terms of IOU, coverage, and end-to-end answer quality. The approach unifies query relevance and intra-set diversity under token constraints, enabling automatic, instance-specific calibration without manual tuning and robust performance across domains. Overall, AdaGReS offers a scalable, theoretically grounded solution for robust context selection in token-budgeted RAG, with strong practical impact for real-world knowledge-intensive tasks.

Abstract

Retrieval-augmented generation (RAG) is highly sensitive to the quality of selected context, yet standard top-k retrieval often returns redundant or near-duplicate chunks that waste token budget and degrade downstream generation. We present AdaGReS, a redundancy-aware context selection framework for token-budgeted RAG that optimizes a set-level objective combining query-chunk relevance and intra-set redundancy penalties. AdaGReS performs greedy selection under a token-budget constraint using marginal gains derived from the objective, and introduces a closed-form, instance-adaptive calibration of the relevance-redundancy trade-off parameter to eliminate manual tuning and adapt to candidate-pool statistics and budget limits. We further provide a theoretical analysis showing that the proposed objective exhibits epsilon-approximate submodularity under practical embedding similarity conditions, yielding near-optimality guarantees for greedy selection. Experiments on open-domain question answering (Natural Questions) and a high-redundancy biomedical (drug) corpus demonstrate consistent improvements in redundancy control and context quality, translating to better end-to-end answer quality and robustness across settings.

Figures (5)

• Figure 1: It illustrates the difference in marginal gain behaviors between strictly submodular objectives and our modular-minus-supermodular objective: strictly submodular objectives see marginal gains that consistently decrease as the set grows, while our modular-minus-supermodular objective may exhibit flat or even increasing marginal gains when high redundancy or non-overlapping candidates are present.
• Figure 2: Visualization of IOU (Intersection over Union) scores between the dynamic $\beta$ method and the baseline method.
• Figure 3: Visualization of Intersection over Union (IOU) Scores between the Dynamic $\beta$ Method and Baseline Methods under Different Redundancy Thresholds
• Figure 4: Comparison for $\beta$ = 0.55, 0.65, 0.7 (greedy vs. simple)
• Figure 5: Comparison for $\beta$ = 0.55, 0.65, 0.7 (greedy vs. simple)