Rethinking the Reranker: Boundary-Aware Evidence Selection for Robust Retrieval-Augmented Generation

TL;DR

This work tackles the brittleness of retrieval-augmented generation (RAG) under realistic retrieval noise by reframing the reranker as a boundary-aware evidence selector that targets the generator’s Goldilocks Zone—evidence that is challenging yet solvable. BAR-RAG uses a two-stage reinforcement learning framework: Stage 1 trains a selector to curate evidence sets near the generator’s competence boundary, and Stage 2 fine-tunes the generator under the induced evidence distribution to close train–test gaps. The approach yields consistent end-to-end QA gains (average +10.3%) across seven knowledge-intensive benchmarks and improves robustness to noisy or partial retrieval without adding inference cost. The methodology advances evidence-quality control in RAG, enabling more reliable knowledge synthesis in multi-hop and fact-based tasks, with publicly available code to facilitate adoption and further study.

Abstract

Retrieval-Augmented Generation (RAG) systems remain brittle under realistic retrieval noise, even when the required evidence appears in the top-K results. A key reason is that retrievers and rerankers optimize solely for relevance, often selecting either trivial, answer-revealing passages or evidence that lacks the critical information required to answer the question, without considering whether the evidence is suitable for the generator. We propose BAR-RAG, which reframes the reranker as a boundary-aware evidence selector that targets the generator's Goldilocks Zone -- evidence that is neither trivially easy nor fundamentally unanswerable for the generator, but is challenging yet sufficient for inference and thus provides the strongest learning signal. BAR-RAG trains the selector with reinforcement learning using generator feedback, and adopts a two-stage pipeline that fine-tunes the generator under the induced evidence distribution to mitigate the distribution mismatch between training and inference. Experiments on knowledge-intensive question answering benchmarks show that BAR-RAG consistently improves end-to-end performance under noisy retrieval, achieving an average gain of 10.3 percent over strong RAG and reranking baselines while substantially improving robustness. Code is publicly avaliable at https://github.com/GasolSun36/BAR-RAG.

Figures (5)

• Figure 1: Comparison between standard relevance-based rerankers and our boundary-aware evidence selection. Standard rerankers maximize relevance scores but overlook the generator’s weaknesses, often encouraging shortcut learning and brittle reasoning by prioritizing trivial or answer-revealing evidence. In contrast, our method selects challenging yet solvable evidence based on generator uncertainty, promoting robust reasoning and reducing the mismatch between the evidence distributions encountered during training and inference under noisy retrieval.
• Figure 2: Recall@5 and Recall@10 vs. QA Accuracy across different datasets. Higher retrieval recall does not guarantee higher QA accuracy. Our method narrows the gap between recall and accuracy.
• Figure 3: Overview of the BAR-RAG training and inference pipeline. During training, we adopt a two-stage framework: (Stage 1) a selector is trained with reinforcement learning using relevance and uncertainty rewards to identify challenging yet solvable evidence sets, and (Stage 2) the generator is optimized under the induced evidence distribution using accuracy, formatting, and citation rewards. At inference time, the trained generator answers questions using retrieved documents, producing robust and high-quality answers.
• Figure 4: Top-$K$ accuracy curves on four QA benchmarks for two base models: Qwen-2.5-7B-Instruct and LLaMA-3.1-8B-Instruct. From left to right, columns correspond to NQ, TriviaQA, PopQA, and HotpotQA. Across both models and all datasets, our method consistently achieves higher accuracy in low-$K$ regimes and remains robust as $K$ increases, whereas standard RAG exhibits weaker scaling behavior and higher sensitivity to retrieval noise.
• Figure 5: Distribution shift of evidence difficulty ($\hat{p}$) across different retrieval methods. Our method concentrates samples toward the target correctness level $c{=}0.5$, substantially reducing extreme cases near 0 (unsolvable) and 1 (trivially solvable).