Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MARAGE: Transferable Multi-Model Adversarial Attack for Retrieval-Augmented Generation Data Extraction

Xiao Hu, Eric Liu, Weizhou Wang, Xiangyu Guo, David Lie

TL;DR

This work tackles the privacy risk of data leakage in Retrieval-Augmented Generation by introducing MARAGE, a transferable, optimization-based RAG extraction attack. MARAGE crafts a universal adversarial string through continuous optimization across multiple models and a primacy weighting scheme to emphasize early target tokens, enabling verbatim leakage of RAG data to unseen models. Empirical results show MARAGE outperforming manual and optimization baselines across diverse RAG datasets and model architectures, with strong transferability and insights from probing analyses. The study also investigates defenses and emphasizes the need for intrinsic model safeguards and adversarial training to mitigate such leakage in practical RAG deployments.

Abstract

Retrieval-Augmented Generation (RAG) offers a solution to mitigate hallucinations in Large Language Models (LLMs) by grounding their outputs to knowledge retrieved from external sources. The use of private resources and data in constructing these external data stores can expose them to risks of extraction attacks, in which attackers attempt to steal data from these private databases. Existing RAG extraction attacks often rely on manually crafted prompts, which limit their effectiveness. In this paper, we introduce a framework called MARAGE for optimizing an adversarial string that, when appended to user queries submitted to a target RAG system, causes outputs containing the retrieved RAG data verbatim. MARAGE leverages a continuous optimization scheme that integrates gradients from multiple models with different architectures simultaneously to enhance the transferability of the optimized string to unseen models. Additionally, we propose a strategy that emphasizes the initial tokens in the target RAG data, further improving the attack's generalizability. Evaluations show that MARAGE consistently outperforms both manual and optimization-based baselines across multiple LLMs and RAG datasets, while maintaining robust transferability to previously unseen models. Moreover, we conduct probing tasks to shed light on the reasons why MARAGE is more effective compared to the baselines and to analyze the impact of our approach on the model's internal state.

MARAGE: Transferable Multi-Model Adversarial Attack for Retrieval-Augmented Generation Data Extraction

TL;DR

This work tackles the privacy risk of data leakage in Retrieval-Augmented Generation by introducing MARAGE, a transferable, optimization-based RAG extraction attack. MARAGE crafts a universal adversarial string through continuous optimization across multiple models and a primacy weighting scheme to emphasize early target tokens, enabling verbatim leakage of RAG data to unseen models. Empirical results show MARAGE outperforming manual and optimization baselines across diverse RAG datasets and model architectures, with strong transferability and insights from probing analyses. The study also investigates defenses and emphasizes the need for intrinsic model safeguards and adversarial training to mitigate such leakage in practical RAG deployments.

Abstract

Retrieval-Augmented Generation (RAG) offers a solution to mitigate hallucinations in Large Language Models (LLMs) by grounding their outputs to knowledge retrieved from external sources. The use of private resources and data in constructing these external data stores can expose them to risks of extraction attacks, in which attackers attempt to steal data from these private databases. Existing RAG extraction attacks often rely on manually crafted prompts, which limit their effectiveness. In this paper, we introduce a framework called MARAGE for optimizing an adversarial string that, when appended to user queries submitted to a target RAG system, causes outputs containing the retrieved RAG data verbatim. MARAGE leverages a continuous optimization scheme that integrates gradients from multiple models with different architectures simultaneously to enhance the transferability of the optimized string to unseen models. Additionally, we propose a strategy that emphasizes the initial tokens in the target RAG data, further improving the attack's generalizability. Evaluations show that MARAGE consistently outperforms both manual and optimization-based baselines across multiple LLMs and RAG datasets, while maintaining robust transferability to previously unseen models. Moreover, we conduct probing tasks to shed light on the reasons why MARAGE is more effective compared to the baselines and to analyze the impact of our approach on the model's internal state.

Paper Structure

This paper contains 33 sections, 11 equations, 15 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Threat Model
  3. The target RAG system
  4. The attacker
  5. Methodology
  6. Adversarial Objective
  7. Relaxing Discrete Optimization
  8. Evaluation
  9. Experiment Settings
  10. Datasets and RAG simulation
  11. Evaluation Metrics
  12. Baseline Methods
  13. RQ1:RAG Extraction Attack Effectiveness
  14. RQ2:Transferability
  15. RQ3:Layer and Token-wise Probing
  16. ...and 18 more sections

Figures (15)

  • Figure 1: The whole workflow of MARAGE on optimizing the universal adversarial embeddings
  • Figure 2: BLEU score and Semantic Similarity(SS) for all three baselines and MARAGE on the five models and Rag-12000.
  • Figure 3: TSNE scatter plot for visualizing the last layer attention outputs for MARAGE, Pleakhui_pleak_2024, and manual attackzeng_good_2024 on different token positions.
  • Figure 4: Impact of the length of the ADV
  • Figure 5: Impact of the size of the $D_p$
  • ...and 10 more figures