Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases

TL;DR

The paper investigates privacy risks in Retrieval-Augmented Generation (RAG) systems by introducing Pirates of the RAG, a fully automated black-box attack that iteratively exposes a private knowledge base. It employs an adaptive, anchor-based strategy guided by a relevance mechanism and powered by attacker-side open-source LLMs and embeddings to craft queries that reveal missing chunks from the hidden KB. Compared to recent baselines, the proposed Pirate algorithm achieves superior coverage and leakage of the private knowledge across multiple RAG configurations, demonstrating significant privacy vulnerabilities. The work highlights urgent needs for robust privacy safeguards in RAG design and deployment and discusses limitations and potential defenses, including guardrails beyond static detectors. The findings have practical implications for the security of real-world RAG deployments and motivate targeted defenses and policy considerations.

Abstract

The growing ubiquity of Retrieval-Augmented Generation (RAG) systems in several real-world services triggers severe concerns about their security. A RAG system improves the generative capabilities of a Large Language Models (LLM) by a retrieval mechanism which operates on a private knowledge base, whose unintended exposure could lead to severe consequences, including breaches of private and sensitive information. This paper presents a black-box attack to force a RAG system to leak its private knowledge base which, differently from existing approaches, is adaptive and automatic. A relevance-based mechanism and an attacker-side open-source LLM favor the generation of effective queries to leak most of the (hidden) knowledge base. Extensive experimentation proves the quality of the proposed algorithm in different RAG pipelines and domains, comparing to very recent related approaches, which turn out to be either not fully black-box, not adaptive, or not based on open-source models. The findings from our study remark the urgent need for more robust privacy safeguards in the design and deployment of RAG systems.

Figures (7)

• Figure 1: Attacking a RAG system with the proposed algorithm, following the "Pirate" metaphor of this paper. The red connectors show how private pieces information (coins) "moves" from the private knowledge (chest) to the attacker (pirate) knowledge base, by convincing the RAG LLM (parrot) to expose them. The attack is generated by means of anchors (paired with relevance), and thanks to an attacker-side LLM and embedder, both based on open-source tools that can run on a domestic computer.
• Figure 2: Evolution of anchor set $\mathcal{A}_t$ during the (unbounded) attack procedure of Algorithm \ref{['alg:algo']}. Dashed curves are about anchors with zero relevance (dead anchors).
• Figure 3: Pale: number of extracted chunks (LC metric) during the attack procedure (bounded case). Opaque: number of unique chunks (ULC metric).
• Figure 4: Distribution of adversarial query (query+command) lengths, measured in the number of words, generated across methods (represented by different colors in the legend) in the bounded setting. The three subplots correspond to three different agents: Agent A, Agent B, and Agent C. Each bar in the histograms represents the frequency of queries of a particular length. Since RThief and RThief* share the same adversarial query generation technique we only show RThief.
• Figure 5: Distribution of the cosine-similarity scores between the top-$k$ retrieved chunks and the adversarial queries, across agents (sub-figures) and methods (colors), in the bounded setting. Since RThief and RThief* share the same adversarial query generation technique we only show RThief.