Unsupervised Corpus Poisoning Attacks in Continuous Space for Dense Retrieval

TL;DR

This work tackles corpus poisoning in dense retrieval by moving adversarial generation from discrete lexical edits to continuous embedding perturbations. It introduces a two-component framework: a reconstruction model that can recover text from contextual embeddings and a perturbation model that crafts adversarial embeddings while preserving embedding proximity to the original, enabling fast and unobtrusive document injections. The authors demonstrate that their unsupervised approach achieves competitive attack effectiveness in both white-box and black-box settings, with lower perplexity and substantially faster generation than previous gradient-based methods, and show potential for adversarial training to improve robustness. The findings highlight practical risks for dense retrieval systems and suggest avenues for defense and detection beyond traditional perplexity-based filtering.

Abstract

This paper concerns corpus poisoning attacks in dense information retrieval, where an adversary attempts to compromise the ranking performance of a search algorithm by injecting a small number of maliciously generated documents into the corpus. Our work addresses two limitations in the current literature. First, attacks that perform adversarial gradient-based word substitution search do so in the discrete lexical space, while retrieval itself happens in the continuous embedding space. We thus propose an optimization method that operates in the embedding space directly. Specifically, we train a perturbation model with the objective of maintaining the geometric distance between the original and adversarial document embeddings, while also maximizing the token-level dissimilarity between the original and adversarial documents. Second, it is common for related work to have a strong assumption that the adversary has prior knowledge about the queries. In this paper, we focus on a more challenging variant of the problem where the adversary assumes no prior knowledge about the query distribution (hence, unsupervised). Our core contribution is an adversarial corpus attack that is fast and effective. We present comprehensive experimental results on both in- and out-of-domain datasets, focusing on two related tasks: a top-1 attack and a corpus poisoning attack. We consider attacks under both a white-box and a black-box setting. Notably, our method can generate successful adversarial examples in under two minutes per target document; four times faster compared to the fastest gradient-based word substitution methods in the literature with the same hardware. Furthermore, our adversarial generation method generates text that is more likely to occur under the distribution of natural text (low perplexity), and is therefore more difficult to detect.

Figures (9)

• Figure 1: Illustration of our unsupervised corpus poisoning attack under our threat model. We attack a retriever's ranking performance by generating uninformative documents with high relevance scores. For example, encoding the original document $d$ and its adversarial counterpart $\tilde{d}$ with SimLM produces similar embeddings, but $\tilde{d}$ is nonsensical.
• Figure 2: The training pipeline for the reconstruction model.
• Figure 3: The pipeline of generating new adversarial documents. The perturbation model is trained using a combined loss to transform the embeddings of target documents into adversarial ones. Then, the trained reconstruction model recovers adversarial embeddings to adversarial documents.
• Figure 4: Definition of Attack Success Rate (ASR). For an arbitrary ranking, if there is an adversarial document $\tilde{d}$ that exceeds the relevant document with the lowest score $d_9$, this ranking is considered as attacked successfully.
• Figure 5: The Top-1 black-box attack results by transferring adversarial documents from SimLM to other retrieval models.