Enhancing Retrieval-Augmented Generation with Two-Stage Retrieval: FlashRank Reranking and Query Expansion
Sherine George
TL;DR
This work tackles the recall versus context limitation in Retrieval-Augmented Generation by introducing FlashRank, a two-stage pipeline that combines LLM-driven query expansion with a marginal-utility reranker under a token budget. It formalizes the recall-utility trade-off and presents a concrete algorithm that greedily selects evidence to maximize information gain per token, while adaptively learning the contribution of relevance, novelty, length, and cross-encoder signals. Empirical results across BEIR, MS MARCO, and FinanceBench show improved retrieval and generation quality, with notable efficiency gains and robustness in finance-related tasks. The proposed approach offers a practical, domain-agnostic solution for grounding LLMs with external evidence at scale.
Abstract
Retrieval-Augmented Generation (RAG) couples a retriever with a large language model (LLM) to ground generated responses in external evidence. While this framework enhances factuality and domain adaptability, it faces a key bottleneck: balancing retrieval recall with limited LLM context. Retrieving too few passages risks missing critical context, while retrieving too many overwhelms the prompt window, diluting relevance and increasing cost. We propose a two-stage retrieval pipeline that integrates LLM-driven query expansion to improve candidate recall and FlashRank, a fast marginal-utility reranker that dynamically selects an optimal subset of evidence under a token budget. FlashRank models document utility as a weighted combination of relevance, novelty, brevity, and cross-encoder evidence. Together, these modules form a generalizable solution that increases answer accuracy, faithfulness, and computational efficiency.