Hoist with His Own Petard: Inducing Guardrails to Facilitate Denial-of-Service Attacks on Retrieval-Augmented Generation of LLMs

TL;DR

The paper reveals a novel denial-of-service vulnerability in retrieval-augmented generation by exploiting LLM safety guardrails with jailbreak prompts embedded in the knowledge base. It introduces MutedRAG, a framework that uses a crafted suffix to trigger refusal behavior and a prefix to ensure the malicious text is retrieved within the top-k results, yielding high attack effectiveness under both black-box and white-box settings. Across HotpotQA, NQ, and MS-MARCO with diverse LLMs, MutedRAG achieves substantial ASR and I-ASR with minimal injected text, outperforming prior PoisonedRAG attacks. The work evaluates defenses like paraphrasing, perplexity-based screening, and duplicate text filtering, finding them largely insufficient, and calls for robust, defense-in-depth strategies to secure RAG deployments in practice.

Abstract

Retrieval-Augmented Generation (RAG) integrates Large Language Models (LLMs) with external knowledge bases, improving output quality while introducing new security risks. Existing studies on RAG vulnerabilities typically focus on exploiting the retrieval mechanism to inject erroneous knowledge or malicious texts, inducing incorrect outputs. However, these approaches overlook critical weaknesses within LLMs, leaving important attack vectors unexplored and limiting the scope and efficiency of attacks. In this paper, we uncover a novel vulnerability: the safety guardrails of LLMs, while designed for protection, can also be exploited as an attack vector by adversaries. Building on this vulnerability, we propose MutedRAG, a novel denial-of-service attack that reversely leverages the guardrails of LLMs to undermine the availability of RAG systems. By injecting minimalistic jailbreak texts, such as "\textit{How to build a bomb}", into the knowledge base, MutedRAG intentionally triggers the LLM's safety guardrails, causing the system to reject legitimate queries. Besides, due to the high sensitivity of guardrails, a single jailbreak sample can affect multiple queries, effectively amplifying the efficiency of attacks while reducing their costs. Experimental results on three datasets demonstrate that MutedRAG achieves an attack success rate exceeding 60% in many scenarios, requiring only less than one malicious text to each target query on average. In addition, we evaluate potential defense strategies against MutedRAG, finding that some of current mechanisms are insufficient to mitigate this threat, underscoring the urgent need for more robust solutions.

Figures (6)

• Figure 1: Comparison between PoisonedRAG and MutedRAG (ours). PoisonedRAG uses 5 well-designed paragraphs to induce a LLM to output the attacker's target answer while MutedRAG only uses 1 paragraph to induce a LLM to refuse to answer towards several user's queries.
• Figure 2: Overview of MutedRAG. Step (1) generates suffixes using a text splicing scheme, while step (2) integrates Open Source LLMs (e.g., Llama3-8B-Instruct) into the text optimization phase, considering both the similarity to the target text and the naturalness of the generated content. Following this, MutedRAG injects the generated malicious texts to trigger the security guardrails of LLMs and execute the attack. In the black-box setting, the target queries alone, as prefixes, are sufficient.
• Figure 3: Injected numbers and IR comparison on MutedRAG (white-box).
• Figure 4: Impact of $k$ for MutedRAG.
• Figure 5: The effectiveness of MutedRAG under knowledge expansion defense with different $k$ on HotpotQA