Connect the Dots: Knowledge Graph-Guided Crawler Attack on Retrieval-Augmented Generation Systems

TL;DR

This work addresses privacy risks in retrieval-augmented generation (RAG) by modeling adversarial data extraction as an Adaptive Stochastic Coverage Problem (ASCP) with a target of maximizing coverage under a query budget $B$. It introduces RAGCrawler, a knowledge-graph-guided attacker that maintains a global attacker KG, a KG-Constructor, a Strategy Scheduler, and a Query Generator to realize a near-optimal adaptive greedy policy based on Conditional Marginal Gain (CMG). The approach demonstrates robustness to defenses like query rewriting and multi-query retrieval, achieving high corpus coverage (average ~66.8%) and strong semantic fidelity across diverse datasets and configurations, and it can produce high-quality surrogate RAG systems from extracted content. The results highlight significant security gaps in current RAG architectures and underscore the urgent need for behavior-aware defenses and query-provenance-based safeguards to protect private knowledge bases against such long-horizon attacks.

Abstract

Retrieval-augmented generation (RAG) systems integrate document retrieval with large language models and have been widely adopted. However, in privacy-related scenarios, RAG introduces a new privacy risk: adversaries can issue carefully crafted queries to exfiltrate sensitive content from the underlying corpus gradually. Although recent studies have demonstrated multi-turn extraction attacks, they rely on heuristics and fail to perform long-term extraction planning. To address these limitations, we formulate the RAG extraction attack as an adaptive stochastic coverage problem (ASCP). In ASCP, each query is treated as a probabilistic action that aims to maximize conditional marginal gain (CMG), enabling principled long-term planning under uncertainty. However, integrating ASCP with practical RAG attack faces three key challenges: unobservable CMG, intractability in the action space, and feasibility constraints. To overcome these challenges, we maintain a global attacker-side state to guide the attack. Building on this idea, we introduce RAGCRAWLER, which builds a knowledge graph to represent revealed information, uses this global state to estimate CMG, and plans queries in semantic space that target unretrieved regions. In comprehensive experiments across diverse RAG architectures and datasets, our proposed method, RAGCRAWLER, consistently outperforms all baselines. It achieves up to 84.4% corpus coverage within a fixed query budget and deliver an average improvement of 20.7% over the top-performing baseline. It also maintains high semantic fidelity and strong content reconstruction accuracy with low attack cost. Crucially, RAGCRAWLER proves its robustness by maintaining effectiveness against advanced RAG systems employing query rewriting and multi-query retrieval strategies. Our work reveals significant security gaps and highlights the pressing need for stronger safeguards for RAG.

Figures (15)

• Figure 1: A motivating example. (a) A data extraction scenario on a private medical corpus. The continuation strategy follows the recent answer’s context, which can drift away from the corpus content; the keyword strategy reuses extracted keywords to form new queries, often yielding redundant information from the same semantic region; our proposed strategy maintains a global graph, detects unrevealed facts and targets them with new queries. (b) Knowledge coverage vs. query number for each strategy on the example corpus. (c) Distribution of retrieved facts in the corpus’ semantic space. Each point is a fact or document, with colors indicating queries from different strategies.
• Figure 2: Illustration of gap detection by the Global Strategy (Ours). The strategy identifies from the graph that the patient has a missing Personal Medical History, and guides the next exploration step to complete the knowledge.
• Figure 3: Threat model of RAG Crawling. An external attacker interacts with a production RAG system only via the public interface. For each query, the retriever selects a set of documents from the private corpus and the LLM generates an answer conditioned on this evidence. Across turns, the attacker aims to stealthily expand the union of leaked retrieved items under a query budget.
• Figure 4: The workflow of RAGCrawler. At each step, (1) The KG-Constructor processes the latest system response to update the knowledge graph. (2) The Strategy Scheduler analyzes this graph to select a strategic anchor. (3) The Query Generator then uses this anchor to formulate the next query, completing the loop.
• Figure 5: The workflow of KG-Constructor. Guided by a Topic-Specific Prior, an Iterative Extraction and Reflection process generates a knowledge subgraph ($\Delta\mathcal{G}_t$), which is then refined through Incremental Graph Update and Semantic Merging to produce the final graph state, $\mathcal{G}_t$.

Theorems & Definitions (5)

• Theorem 1: Approximation guarantee of adaptive greedy nemhauser1978analysiswolsey1982analysiskrause2007neargolovin2011adaptivekhuller1999budgeted
• Definition 1: Conditional Marginal Gain in RAG Crawling
• Theorem 2: Adaptive Monotonicity and Submodularity in RAG Crawling
• Theorem 3: Near-optimality guarantee of RAGCrawler
• proof