Blending Learning to Rank and Dense Representations for Efficient and Effective Cascades
Franco Maria Nardini, Raffaele Perego, Nicola Tonellotto, Salvatore Trani
TL;DR
This work tackles ad-hoc passage retrieval by integrating lexical sparse features with dense neural representations in a two-stage pipeline. A dense first-stage retriever (STAR/CONTRIEVER) identifies candidate documents, which are then re-ranked by a LambdaMART-based LTR model that blends 2,559 features derived from dense embeddings, their similarity, and 253 hand-crafted lexical cues. Across MS-MARCO and DL19/DL20 datasets, the full-featured re-ranker achieves up to $nDCG@10$ gains of $11\%$ with only about a $4.3\%$ increase in end-to-end latency, while remaining CPU-efficient compared to cross-encoder baselines. The results demonstrate that lexical and neural signals offer complementary information and that a learned, tree-based fusion can improve retrieval effectiveness without sacrificing scalability.
Abstract
We investigate the exploitation of both lexical and neural relevance signals for ad-hoc passage retrieval. Our exploration involves a large-scale training dataset in which dense neural representations of MS-MARCO queries and passages are complemented and integrated with 253 hand-crafted lexical features extracted from the same corpus. Blending of the relevance signals from the two different groups of features is learned by a classical Learning-to-Rank (LTR) model based on a forest of decision trees. To evaluate our solution, we employ a pipelined architecture where a dense neural retriever serves as the first stage and performs a nearest-neighbor search over the neural representations of the documents. Our LTR model acts instead as the second stage that re-ranks the set of candidates retrieved by the first stage to enhance effectiveness. The results of reproducible experiments conducted with state-of-the-art dense retrievers on publicly available resources show that the proposed solution significantly enhances the end-to-end ranking performance while relatively minimally impacting efficiency. Specifically, we achieve a boost in nDCG@10 of up to 11% with an increase in average query latency of only 4.3%. This confirms the advantage of seamlessly combining two distinct families of signals that mutually contribute to retrieval effectiveness.