Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generation

Kennedy Edemacu, Vinay M. Shashidhar, Micheal Tuape, Dan Abudu, Beakcheol Jang, Jong Wook Kim

Abstract

Retrieval-Augmented Generation (RAG) has emerged as a powerful approach to boost the capabilities of large language models (LLMs) by incorporating external, up-to-date knowledge sources. However, this introduces a potential vulnerability to knowledge poisoning attacks, where attackers can compromise the knowledge source to mislead the generation model. One such attack is the PoisonedRAG in which the injected adversarial texts steer the model to generate an attacker-chosen response to a target question. In this work, we propose novel defense methods, FilterRAG and ML-FilterRAG, to mitigate the PoisonedRAG attack. First, we propose a new property to uncover distinct properties to differentiate between adversarial and clean texts in the knowledge data source. Next, we employ this property to filter out adversarial texts from clean ones in the design of our proposed approaches. Evaluation of these methods using benchmark datasets demonstrate their effectiveness, with performances close to those of the original RAG systems.

Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generation

Abstract

Retrieval-Augmented Generation (RAG) has emerged as a powerful approach to boost the capabilities of large language models (LLMs) by incorporating external, up-to-date knowledge sources. However, this introduces a potential vulnerability to knowledge poisoning attacks, where attackers can compromise the knowledge source to mislead the generation model. One such attack is the PoisonedRAG in which the injected adversarial texts steer the model to generate an attacker-chosen response to a target question. In this work, we propose novel defense methods, FilterRAG and ML-FilterRAG, to mitigate the PoisonedRAG attack. First, we propose a new property to uncover distinct properties to differentiate between adversarial and clean texts in the knowledge data source. Next, we employ this property to filter out adversarial texts from clean ones in the design of our proposed approaches. Evaluation of these methods using benchmark datasets demonstrate their effectiveness, with performances close to those of the original RAG systems.

Paper Structure

This paper contains 39 sections, 6 equations, 4 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Threat Model
  4. The Knowledge Corruption Attack
  5. Motivation
  6. Methodology
  7. Overview
  8. Filtration of Adversarial Texts
  9. FilterRAG (Threshold-Based Approach)
  10. ML-FilterRAG (Machine Learning Approach)
  11. Experiments
  12. Datasets
  13. RAG Settings
  14. Retriever
  15. SLM
  16. ...and 24 more sections

Figures (4)

  • Figure 1: Pair Plot for Freq-Density vs Perplexity
  • Figure 2: A high-level illustration of our framework. A filtration phase is integrated into the tradition RAG components to filter-out adversarial texts before the generation phase.
  • Figure 3: The prompt template used to produce the results zou2024poisonedrag
  • Figure 4: Pair plots for all the features