Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Secure Retrieval-Augmented Generation against Poisoning Attacks

Zirui Cheng, Jikai Sun, Anjun Gao, Yueyang Quan, Zhuqing Liu, Xiaohua Hu, Minghong Fang

TL;DR

This work tackles poisoning attacks on Retrieval-Augmented Generation by introducing RAGuard, which first expands the retrieved set to increase clean content and then applies a three-filter defense. The filters combine chunk-wise perplexity, perplexity maxima, and text similarity in a non-parametric, empirical-threshold framework to detect and remove poisoned texts prior to generation. The approach yields strong detection and output accuracy across five datasets, including against adaptive attacks, while maintaining low false positives and modest computational overhead. By providing a formal performance bound and extensive empirical validation, RAGuard offers a practical, robust defense for knowledge-grounded NLP systems in high-stakes domains.

Abstract

Large language models (LLMs) have transformed natural language processing (NLP), enabling applications from content generation to decision support. Retrieval-Augmented Generation (RAG) improves LLMs by incorporating external knowledge but also introduces security risks, particularly from data poisoning, where the attacker injects poisoned texts into the knowledge database to manipulate system outputs. While various defenses have been proposed, they often struggle against advanced attacks. To address this, we introduce RAGuard, a detection framework designed to identify poisoned texts. RAGuard first expands the retrieval scope to increase the proportion of clean texts, reducing the likelihood of retrieving poisoned content. It then applies chunk-wise perplexity filtering to detect abnormal variations and text similarity filtering to flag highly similar texts. This non-parametric approach enhances RAG security, and experiments on large-scale datasets demonstrate its effectiveness in detecting and mitigating poisoning attacks, including strong adaptive attacks.

Secure Retrieval-Augmented Generation against Poisoning Attacks

TL;DR

This work tackles poisoning attacks on Retrieval-Augmented Generation by introducing RAGuard, which first expands the retrieved set to increase clean content and then applies a three-filter defense. The filters combine chunk-wise perplexity, perplexity maxima, and text similarity in a non-parametric, empirical-threshold framework to detect and remove poisoned texts prior to generation. The approach yields strong detection and output accuracy across five datasets, including against adaptive attacks, while maintaining low false positives and modest computational overhead. By providing a formal performance bound and extensive empirical validation, RAGuard offers a practical, robust defense for knowledge-grounded NLP systems in high-stakes domains.

Abstract

Large language models (LLMs) have transformed natural language processing (NLP), enabling applications from content generation to decision support. Retrieval-Augmented Generation (RAG) improves LLMs by incorporating external knowledge but also introduces security risks, particularly from data poisoning, where the attacker injects poisoned texts into the knowledge database to manipulate system outputs. While various defenses have been proposed, they often struggle against advanced attacks. To address this, we introduce RAGuard, a detection framework designed to identify poisoned texts. RAGuard first expands the retrieval scope to increase the proportion of clean texts, reducing the likelihood of retrieving poisoned content. It then applies chunk-wise perplexity filtering to detect abnormal variations and text similarity filtering to flag highly similar texts. This non-parametric approach enhances RAG security, and experiments on large-scale datasets demonstrate its effectiveness in detecting and mitigating poisoning attacks, including strong adaptive attacks.

Paper Structure

This paper contains 23 sections, 1 theorem, 8 equations, 6 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries and Related Work
  3. Retrieval-Augmented Generation (RAG)
  4. Poisoning Attacks and Defenses to RAG
  5. Problem Formulation
  6. Threat model
  7. Defense objectives
  8. Our RAGuard
  9. Overview
  10. Chunk-wise perplexity filtering
  11. Text similarity filtering
  12. Formal Security Analysis
  13. Experimental Setup
  14. Datasets
  15. Poisoning Attacks
  16. ...and 8 more sections

Key Result

Theorem 1

Suppose the RAG system retrieves a final set of $k$ texts after applying RAGuard filtering, and assume the LLM answers correctly whenever more than half of these $k$ texts are benign. Let $\rho$ be the fraction of poisoned texts in the knowledge database, and let $\beta_{\mathrm{PD}}, \beta_{\mathrm where $c = \tfrac{1}{3}\,(\tfrac{1}{2} - \rho \beta_{\mathrm{total}})^2 \rho \beta_{\mathrm{total}}

Figures (6)

  • Figure 1: Illustration of the RAG process.
  • Figure 2: Impact of $k$ on NQ dataset.
  • Figure 3: Impact of $N$ on NQ dataset.
  • Figure 4: Impact of poisoned texts per query on NQ dataset.
  • Figure 5: Number of related clean texts of each query.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof