Exploiting Positional Bias for Query-Agnostic Generative Content in Search

TL;DR

A simple yet effective compensation for the weaknesses of the NRMs in search is investigated, validating the hypotheses regarding transformer bias and finding that contextualisation of a non-relevant text further reduces negative effects whilst likely circumventing existing content filtering mechanisms.

Abstract

In recent years, neural ranking models (NRMs) have been shown to substantially outperform their lexical counterparts in text retrieval. In traditional search pipelines, a combination of features leads to well-defined behaviour. However, as neural approaches become increasingly prevalent as the final scoring component of engines or as standalone systems, their robustness to malicious text and, more generally, semantic perturbation needs to be better understood. We posit that the transformer attention mechanism can induce exploitable defects through positional bias in search models, leading to an attack that could generalise beyond a single query or topic. We demonstrate such defects by showing that non-relevant text--such as promotional content--can be easily injected into a document without adversely affecting its position in search results. Unlike previous gradient-based attacks, we demonstrate these biases in a query-agnostic fashion. In doing so, without the knowledge of topicality, we can still reduce the negative effects of non-relevant content injection by controlling injection position. Our experiments are conducted with simulated on-topic promotional text automatically generated by prompting LLMs with topical context from target documents. We find that contextualisation of a non-relevant text further reduces negative effects whilst likely circumventing existing content filtering mechanisms. In contrast, lexical models are found to be more resilient to such content injection attacks. We then investigate a simple yet effective compensation for the weaknesses of the NRMs in search, validating our hypotheses regarding transformer bias.

Figures (5)

• Figure 1: Injection of static and contextualised (document-conditioned by Llama 2) text into a document. BM25 penalises both injections as lexical models estimate relevance in an axiomatic fashion; document length has increased with insufficient additional relevant information. However, the NRM (monoT5) is invariant to the addition of contextualised promotional text. Further examples are enlisted in Appendix \ref{['sec:promo-exam']}.
• Figure 2: Average MRS by injected token distance to salient sequences for monoT5, observe noted reduction in variance near a salient sequence and clear reduction in penalty for non-relevant text after a salient sequence.
• Figure 3: Example generated texts with an example of success (green context) and failure (red context) for each item; the relevant text has been manually annotated in green. Further examples are enlisted in Appendix \ref{['sec:promo-exam']}.
• Figure 4: Sensitivity to $\alpha$ measuring nDCG@10 for position absolute 'after'.
• Figure 5: Sensitivity to $\alpha$ measuring nDCG@10 for position absolute 'after'.