Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation

TL;DR

This paper tackles the privacy and intellectual-property risks posed by knowledge-extraction attacks on Retrieval-Augmented Generation (RAG) systems by proposing the first comprehensive benchmark with standardized protocols. It defines a unified design space spanning RAG architectures, attack and defense modalities, knowledge-base construction, and evaluation metrics, and validates this space through extensive experiments across multiple datasets and model families. Key findings show that effective extraction demands coordinated optimization at both the retrieval and generation stages, while defenses are most effective when layered (thresholds, system prompts, summarization, and input blocking) but no single defense is sufficient. The work highlights important factors such as embedding-model transferability, knowledge indexing formats, and query diversity, offering actionable guidance for building privacy-preserving RAG systems and enabling reproducible cross-study comparisons.

Abstract

Retrieval-Augmented Generation (RAG) has become a cornerstone of knowledge-intensive applications, including enterprise chatbots, healthcare assistants, and agentic memory management. However, recent studies show that knowledge-extraction attacks can recover sensitive knowledge-base content through maliciously crafted queries, raising serious concerns about intellectual property theft and privacy leakage. While prior work has explored individual attack and defense techniques, the research landscape remains fragmented, spanning heterogeneous retrieval embeddings, diverse generation models, and evaluations based on non-standardized metrics and inconsistent datasets. To address this gap, we introduce the first systematic benchmark for knowledge-extraction attacks on RAG systems. Our benchmark covers a broad spectrum of attack and defense strategies, representative retrieval embedding models, and both open- and closed-source generators, all evaluated under a unified experimental framework with standardized protocols across multiple datasets. By consolidating the experimental landscape and enabling reproducible, comparable evaluation, this benchmark provides actionable insights and a practical foundation for developing privacy-preserving RAG systems in the face of emerging knowledge extraction threats. Our code is available here.

Figures (12)

• Figure 1: Knowledge extraction attack on RAG causes privacy/proprietary risks across pervasive high-stake domains.
• Figure 2: (a) Design Space of Knowledge Extraction Attack and Defense Benchmark in RAG systems, including 1) Attack Query Design, 2) Knowledge Base Setup, 3) Defense Strategies, 4) Retrieval/Generator Models, and 5) Evaluation Protocols. (b) Constructing the final generator prompt from system and user messages, with malicious queries and retrieved contexts.
• Figure 3: We compare six knowledge-extraction attacks under four defenses across five metrics, averaged over four datasets. Detailed per-dataset results are in Table \ref{['tab-main']} of Appendix \ref{['app-main']}. Transparent bars in all subfigures are identical, representing attack performance without any defense. The $\text{EE}_{LS}$ evaluation results are omitted for brevity since they mirror the trend of $\text{EE}_{SS}$.
• Figure 4: Effects of different retriever and attacker embedding models on Enron. (Off) Diagonal - (Black)White Box.
• Figure 5: Impacts of Thresholds in Threshold defense. Left: Impact of thresholds. Right: Distribution of top-K retrieval scores for each attacker on HealthCareMagic.