Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Traceback of Poisoning Attacks to Retrieval-Augmented Generation

Baolei Zhang, Haoran Xin, Minghong Fang, Zhuqing Liu, Biao Yi, Tong Li, Zheli Liu

TL;DR

This paper tackles poisoning attacks in retrieval-augmented generation (RAG) by introducing RAGForensics, the first traceback framework that identifies poisoned texts in the knowledge base responsible for attacker-desired outputs. It combines iterative retrieval with LLM-guided, structured prompts (including chain-of-thought) to classify candidate texts as poisoned, updating the knowledge base until the top-$K$ retrieved texts are all benign. The authors also address non-poisoned feedback and propose benign text enhancement (BTE) to improve robustness in such cases, alongside an evaluation against adaptive attacks. Experimental results across three datasets and multiple attacks show high traceback accuracy (DACC > 97.4%), low false positive/false negative rates, and strong resilience against adaptive strategies, highlighting the practical potential of forensics-based defenses in securing RAG systems.

Abstract

Large language models (LLMs) integrated with retrieval-augmented generation (RAG) systems improve accuracy by leveraging external knowledge sources. However, recent research has revealed RAG's susceptibility to poisoning attacks, where the attacker injects poisoned texts into the knowledge database, leading to attacker-desired responses. Existing defenses, which predominantly focus on inference-time mitigation, have proven insufficient against sophisticated attacks. In this paper, we introduce RAGForensics, the first traceback system for RAG, designed to identify poisoned texts within the knowledge database that are responsible for the attacks. RAGForensics operates iteratively, first retrieving a subset of texts from the database and then utilizing a specially crafted prompt to guide an LLM in detecting potential poisoning texts. Empirical evaluations across multiple datasets demonstrate the effectiveness of RAGForensics against state-of-the-art poisoning attacks. This work pioneers the traceback of poisoned texts in RAG systems, providing a practical and promising defense mechanism to enhance their security. Our code is available at: https://github.com/zhangbl6618/RAG-Responsibility-Attribution

Traceback of Poisoning Attacks to Retrieval-Augmented Generation

TL;DR

This paper tackles poisoning attacks in retrieval-augmented generation (RAG) by introducing RAGForensics, the first traceback framework that identifies poisoned texts in the knowledge base responsible for attacker-desired outputs. It combines iterative retrieval with LLM-guided, structured prompts (including chain-of-thought) to classify candidate texts as poisoned, updating the knowledge base until the top- retrieved texts are all benign. The authors also address non-poisoned feedback and propose benign text enhancement (BTE) to improve robustness in such cases, alongside an evaluation against adaptive attacks. Experimental results across three datasets and multiple attacks show high traceback accuracy (DACC > 97.4%), low false positive/false negative rates, and strong resilience against adaptive strategies, highlighting the practical potential of forensics-based defenses in securing RAG systems.

Abstract

Large language models (LLMs) integrated with retrieval-augmented generation (RAG) systems improve accuracy by leveraging external knowledge sources. However, recent research has revealed RAG's susceptibility to poisoning attacks, where the attacker injects poisoned texts into the knowledge database, leading to attacker-desired responses. Existing defenses, which predominantly focus on inference-time mitigation, have proven insufficient against sophisticated attacks. In this paper, we introduce RAGForensics, the first traceback system for RAG, designed to identify poisoned texts within the knowledge database that are responsible for the attacks. RAGForensics operates iteratively, first retrieving a subset of texts from the database and then utilizing a specially crafted prompt to guide an LLM in detecting potential poisoning texts. Empirical evaluations across multiple datasets demonstrate the effectiveness of RAGForensics against state-of-the-art poisoning attacks. This work pioneers the traceback of poisoned texts in RAG systems, providing a practical and promising defense mechanism to enhance their security. Our code is available at: https://github.com/zhangbl6618/RAG-Responsibility-Attribution
Paper Structure (28 sections, 7 equations, 4 figures, 6 tables, 1 algorithm)

This paper contains 28 sections, 7 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries and Related Work
  3. RAG Overview
  4. Poisoning Attacks to RAG
  5. Defenses against Poisoning Attacks to RAG
  6. Traceback of Poisoned Texts in Poisoning Attacks to RAG
  7. Threat Model
  8. Design Challenges
  9. Our Traceback System: RAGForensics
  10. Overview
  11. RAGForensics
  12. Experiments
  13. Experiments
  14. Experimental Setup
  15. Datasets
  16. ...and 13 more sections

Figures (4)

  • Figure 1: The example scenario of our traceback system. (a) an attacker injects poisoned texts into the knowledge database; (b) a user reports feedback that includes the query and the incorrect output; (c) our traceback system RAGForensics identifies poisoned texts based on the user's feedback.
  • Figure 2: Impact of the number of poisoned texts for each targeted query on NQ dataset.
  • Figure 3: Impact of the number of poisoned texts for each targeted query on HotpotQA dataset.
  • Figure 4: Impact of the number of poisoned texts for each targeted query on MS-MARCO dataset.