Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Competitive Retrieval: Going Beyond the Single Query

Haya Nachimovsky, Moshe Tennenholtz, Fiana Raiber, Oren Kurland

TL;DR

This work analyzes competitive retrieval where publishers modify documents to rank highly for multiple queries. Using a game-theoretic model, it shows that equilibria may not exist in multi-query settings, and best-response dynamics need external intervention to stabilize. It complements theory with four ranking competitions employing both feature-based (LambdaMART) and neural (BERT) rankers, including AI-assisted document modification, revealing that neural rankers produce more diverse cross-query rankings and that AI tools influence content quality and strategy. Additionally, it introduces a predictive framework using cross-query features to forecast which non-winning document will become the new winner, demonstrating improved predictive power when leveraging information from other queries of the same topic. The work contributes theoretical insights, empirical evidence from multi-query competitions, and a practical predictor, with a public dataset to enable further research.

Abstract

Previous work on the competitive retrieval setting focused on a single-query setting: document authors manipulate their documents so as to improve their future ranking for a given query. We study a competitive setting where authors opt to improve their document's ranking for multiple queries. We use game theoretic analysis to prove that equilibrium does not necessarily exist. We then empirically show that it is more difficult for authors to improve their documents' rankings for multiple queries with a neural ranker than with a state-of-the-art feature-based ranker. We also present an effective approach for predicting the document most highly ranked in the next induced ranking.

Competitive Retrieval: Going Beyond the Single Query

TL;DR

This work analyzes competitive retrieval where publishers modify documents to rank highly for multiple queries. Using a game-theoretic model, it shows that equilibria may not exist in multi-query settings, and best-response dynamics need external intervention to stabilize. It complements theory with four ranking competitions employing both feature-based (LambdaMART) and neural (BERT) rankers, including AI-assisted document modification, revealing that neural rankers produce more diverse cross-query rankings and that AI tools influence content quality and strategy. Additionally, it introduces a predictive framework using cross-query features to forecast which non-winning document will become the new winner, demonstrating improved predictive power when leveraging information from other queries of the same topic. The work contributes theoretical insights, empirical evidence from multi-query competitions, and a practical predictor, with a public dataset to enable further research.

Abstract

Previous work on the competitive retrieval setting focused on a single-query setting: document authors manipulate their documents so as to improve their future ranking for a given query. We study a competitive setting where authors opt to improve their document's ranking for multiple queries. We use game theoretic analysis to prove that equilibrium does not necessarily exist. We then empirically show that it is more difficult for authors to improve their documents' rankings for multiple queries with a neural ranker than with a state-of-the-art feature-based ranker. We also present an effective approach for predicting the document most highly ranked in the next induced ranking.
Paper Structure (19 sections, 5 theorems, 3 figures, 3 tables)

This paper contains 19 sections, 5 theorems, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Game Theoretic Analysis
  4. Game Definition
  5. Game Analysis
  6. Best response dynamics
  7. Ranking Competitions
  8. Empirical Analysis
  9. Analysis of Strategies
  10. Ranking Functions
  11. Generative AI Tools
  12. Predicting Winners
  13. Setup
  14. Results
  15. Conclusions
  16. ...and 4 more sections

Key Result

lemma 1

If $p \le \frac{1}{m}$, then the profile where all players write $d = (p, \ldots, p)$ is a pure Nash equilibrium.

Figures (3)

  • Figure 1: Averaged feature values of documents that lost in at least four consecutive rounds before winning. The documents are grouped based on whether their feature values four rounds before the win were lower than or equal to ($L \le W_q$) or higher than ($L > W_q$) the values of the winners for the given query. $W_q$ and $W_t$: the averaged feature values of the corresponding winner per query and all the winners per the same topic, respectively. x-axis: the (negative) number of rounds leading up to a win. Note: in NEU$\wedge$AI and NEU$\wedge\urcorner$AI, when BERT is used as a standalone ranking function, the feature values of the losing documents are always lower than those of the winning document; therefore, $L > W_q$ is not shown.
  • Figure 2: Average RBO similarity between all pairs of rankings induced per topic, averaged over topics per round.
  • Figure 3: Average similarity between all documents in a round per topic and (i) Winners: documents that won at least one query for the topic in the previous round and (ii) Losers: all other documents in the previous round.

Theorems & Definitions (5)

  • lemma 1
  • lemma 2
  • theorem 1
  • theorem 2
  • corollary 1