Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attack

Cheng Wang, Yiwei Wang, Yujun Cai, Bryan Hooi

TL;DR

This work tackles the security of retrieval-augmented generation (RAG) systems by addressing corpus poisoning in a black-box setting. It introduces Dynamic Importance-Guided Genetic Algorithm (DIGA), which exploits two retriever properties—insensitivity to token order and bias toward influential tokens—by guiding a TF-IDF–based importance score through a genetically evolving adversarial passage that maximizes cosine similarity to the centroid of query embeddings. DIGA achieves competitive attack success with significantly lower time and memory demands than gradient-based or inversion-based baselines, and demonstrates favorable scalability and transferability across datasets and retrievers. The study highlights critical vulnerabilities in RAG pipelines and motivates the development of robust defenses, as well as safe-offensive research avenues like indirect prompt injection analyses and ethical considerations.

Abstract

Retrieval-augmented generation (RAG) systems enhance large language models by incorporating external knowledge, addressing issues like outdated internal knowledge and hallucination. However, their reliance on external knowledge bases makes them vulnerable to corpus poisoning attacks, where adversarial passages can be injected to manipulate retrieval results. Existing methods for crafting such passages, such as random token replacement or training inversion models, are often slow and computationally expensive, requiring either access to retriever's gradients or large computational resources. To address these limitations, we propose Dynamic Importance-Guided Genetic Algorithm (DIGA), an efficient black-box method that leverages two key properties of retrievers: insensitivity to token order and bias towards influential tokens. By focusing on these characteristics, DIGA dynamically adjusts its genetic operations to generate effective adversarial passages with significantly reduced time and memory usage. Our experimental evaluation shows that DIGA achieves superior efficiency and scalability compared to existing methods, while maintaining comparable or better attack success rates across multiple datasets.

Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attack

TL;DR

This work tackles the security of retrieval-augmented generation (RAG) systems by addressing corpus poisoning in a black-box setting. It introduces Dynamic Importance-Guided Genetic Algorithm (DIGA), which exploits two retriever properties—insensitivity to token order and bias toward influential tokens—by guiding a TF-IDF–based importance score through a genetically evolving adversarial passage that maximizes cosine similarity to the centroid of query embeddings. DIGA achieves competitive attack success with significantly lower time and memory demands than gradient-based or inversion-based baselines, and demonstrates favorable scalability and transferability across datasets and retrievers. The study highlights critical vulnerabilities in RAG pipelines and motivates the development of robust defenses, as well as safe-offensive research avenues like indirect prompt injection analyses and ethical considerations.

Abstract

Retrieval-augmented generation (RAG) systems enhance large language models by incorporating external knowledge, addressing issues like outdated internal knowledge and hallucination. However, their reliance on external knowledge bases makes them vulnerable to corpus poisoning attacks, where adversarial passages can be injected to manipulate retrieval results. Existing methods for crafting such passages, such as random token replacement or training inversion models, are often slow and computationally expensive, requiring either access to retriever's gradients or large computational resources. To address these limitations, we propose Dynamic Importance-Guided Genetic Algorithm (DIGA), an efficient black-box method that leverages two key properties of retrievers: insensitivity to token order and bias towards influential tokens. By focusing on these characteristics, DIGA dynamically adjusts its genetic operations to generate effective adversarial passages with significantly reduced time and memory usage. Our experimental evaluation shows that DIGA achieves superior efficiency and scalability compared to existing methods, while maintaining comparable or better attack success rates across multiple datasets.

Paper Structure

This paper contains 35 sections, 4 equations, 4 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Attacks on RAG Systems and Corpus Poisoning.
  4. Genetic Algorithms in NLP.
  5. Method
  6. Problem Formulation
  7. Motivation
  8. Dynamic Importance-Guided Genetic Algorithm (DIGA)
  9. Token Importance Calculation
  10. Population Initialization
  11. Fitness Evaluation
  12. Genetic Policies
  13. Selection.
  14. Crossover.
  15. Mutation.
  16. ...and 20 more sections

Figures (4)

  • Figure 1: Illustration of our motivations. Top: Demonstrating insensitivity to token order, where cosine similarity remains nearly unchanged after permuting the tokens. Bottom: Highlighting bias towards influential tokens, shown by the varying effects on cosine similarity when different tokens are deleted. Some tokens are more influential since deleting them results in a larger change in similarity.
  • Figure 2: An overview of our proposed method.
  • Figure 3: Scalability Analysis. Note that the HotFlip method is too computationally expensive to generate 50 more adversarial passages.
  • Figure 4: Log-perplexity Distribution. This figure presents the log-perplexity measurements for 50 adversarial passages generated on NFCorpus using the GTR-base model. Methods displayed to the right of the dotted line represent discrete optimization techniques.