Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains

TL;DR

METEORA tackles the opacity and vulnerability of traditional RAG in high-stakes domains by replacing top-$k$ re-ranking with rationale-driven, adaptive evidence selection. It combines a DPO-trained rationale generator, an elbow-detection-based Evidence Chunk Selection Engine (ECSE) for adaptive evidence cutoff $k^*$, and a Verifier LLM to filter poisoned content before generation. Across six datasets in legal, financial, and academic domains, METEORA yields significant improvements in recall and precision, reduces the required evidential context by about $80\%$, and enhances adversarial robustness with a substantial F1 uplift. The work demonstrates that interpretability and security can be synergistic with efficient, evidence-driven retrieval, offering practical benefits for auditing and compliance in sensitive applications.

Abstract

In sensitive domains, Retrieval-Augmented Generation (RAG) must be interpretable and robust because errors do not just mislead, they invite lawsuits, undermine scholarly credibility, and breach compliance. Stakeholders require traceable evidence, clear rationales for why specific evidence is selected, and safeguards against poisoned or misleading content. Yet current RAG pipelines rely on similarity-based retrieval with arbitrary top-k cutoffs, provide no explanation for selections, and remain vulnerable to poisoning attacks. We propose METEORA, which replaces these drawbacks with rationale-driven selection, using explicit reasoning to guide evidence choice, explain decisions, and improve robustness to RAG poisoning. METEORA operates in three stages: (1) a general-purpose LLM is preference-tuned to generate query-conditioned rationales using direct preference optimization; (2) these rationales drive an Evidence Chunk Selection Engine that pairs rationales with retrieved evidence for query-specific relevance and applies elbow detection to choose an adaptive cutoff (optionally expanding context with neighboring chunks); and (3) a Verifier LLM uses the rationales to detect and filter poisoned or misleading evidence before generation. Across six datasets, METEORA achieves 13.41% higher recall and, without expansion, 21.05% higher precision than the strongest baseline. It reduces the evidence needed for comparable recall by 80%, improving downstream answer accuracy by 33.34%, and strengthens adversarial defense by increasing F1 from 0.10 to 0.44. Code is available at: https://anonymous.4open.science/r/METEORA-DC46/README.md

Figures (6)

• Figure 1: METEORA achieves interpretable and robust evidence selection where existing approaches fail. On a challenging legal query from Merger Agreement Understanding Dataset with poisoned evidence, traditional re-rankers select contaminated content producing factually incorrect answers, while LLM-based re-rankers fail due to context length limitations. Only METEORA uses interpretable rationales to explain evidence selection and robustness with verification to filter poisoned content, achieving both interpretability and correctness.
• Figure 2: Overview of our METEORA framework.
• Figure 3: Generated Rationale
• Figure 4: Response accuracy when retrieved evidence matches ground-truth, with variance bars indicating statistical significance (p < 0.01, paired t-test) across trials.
• Figure 5: Positive example demonstrating how METEORA links rationale-based evidence selection (interpretability) with rationale-grounded answer generation (explainability), resulting in a correct and traceable response to a legal question.

Theorems & Definitions (1)

• proof