Quam: Adaptive Retrieval through Query Affinity Modelling

TL;DR

The paper tackles the recall limitation of first-stage retrieval by introducing Quam, a query-affinity modelling approach that learns a document affinity graph with edge weights (Laff) to guide adaptive neighbor expansion during re-ranking. It constructs G_a from an initial corpus graph G_c using a cross-encoder to estimate co-relevance between document pairs and defines a SetAff scoring mechanism to prioritize frontier documents during adaptive retrieval. Empirical results on TREC-DL19/20 show Quam achieves up to 26% recall improvement over standard baselines and up to 12% over Gar, with additional gains observed when using the learnt affinity graph to enhance baseline adaptive methods. The work also demonstrates that Quam can be integrated with existing adaptive retrieval pipelines with modest latency overhead, and provides code for reproducibility, highlighting practical impact for recall-critical, low-latency information retrieval tasks.

Abstract

Building relevance models to rank documents based on user information needs is a central task in information retrieval and the NLP community. Beyond the direct ad-hoc search setting, many knowledge-intense tasks are powered by a first-stage retrieval stage for context selection, followed by a more involved task-specific model. However, most first-stage ranking stages are inherently limited by the recall of the initial ranking documents. Recently, adaptive re-ranking techniques have been proposed to overcome this issue by continually selecting documents from the whole corpus, rather than only considering an initial pool of documents. However, so far these approaches have been limited to heuristic design choices, particularly in terms of the criteria for document selection. In this work, we propose a unifying view of the nascent area of adaptive retrieval by proposing, Quam, a \textit{query-affinity model} that exploits the relevance-aware document similarity graph to improve recall, especially for low re-ranking budgets. Our extensive experimental evidence shows that our proposed approach, Quam improves the recall performance by up to 26\% over the standard re-ranking baselines. Further, the query affinity modelling and relevance-aware document graph modules can be injected into any adaptive retrieval approach. The experimental results show the existing adaptive retrieval approach improves recall by up to 12\%. The code of our work is available at \url{https://github.com/Mandeep-Rathee/quam}.

Figures (11)

• Figure 1: Recall comparison on TREC DL20 dataset when the number of neighbors vary.
• Figure 2: An overview of the adaptive retrieval through the query affinity modelling Quam. The $W_i$'s represent the affinity or edge weights.
• Figure 3: Effect of Learnt Affinity (Laff) scores on adaptive retrieval on DL19 dataset.
• Figure 4: Effect of Learnt Affinity (Laff) scores on adaptive retrieval on DL20 dataset.
• Figure 5: Recall comparison on TREC DL20 dataset when the number of neighbours $k$ (with fixed $b=16$) and batch size $b$ (with fixed $k=16$) vary. The vertical line at $b=64$ separates the region where $b>c$.