CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Models

TL;DR

This work introduces CPA-RAG, a covert black-box poisoning framework that targets Retrieval-Augmented Generation (RAG) systems by generating query-relevant adversarial texts that manipulate both retrieval and generation to produce predefined target answers. It formalizes three attack conditions—Retriever, Generation, and Concealment—and implements a three-stage pipeline: information collection, initialization of malicious texts, and cross-model optimization with retriever-aware filtering. Across diverse datasets, LLMs, and retrievers, CPA-RAG achieves high attack success rates (e.g., $\text{ASR}_{k=5} \approx 0.92$) and strong concealment, outperforming existing baselines by substantial margins and even compromising a commercial BaiLian RAG deployment. The results reveal critical defense gaps and underscore the need for RAG-specific robustness measures to mitigate covert adversarial manipulation in real-world systems.

Abstract

Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by incorporating external knowledge, but its openness introduces vulnerabilities that can be exploited by poisoning attacks. Existing poisoning methods for RAG systems have limitations, such as poor generalization and lack of fluency in adversarial texts. In this paper, we propose CPA-RAG, a black-box adversarial framework that generates query-relevant texts capable of manipulating the retrieval process to induce target answers. The proposed method integrates prompt-based text generation, cross-guided optimization through multiple LLMs, and retriever-based scoring to construct high-quality adversarial samples. We conduct extensive experiments across multiple datasets and LLMs to evaluate its effectiveness. Results show that the framework achieves over 90\% attack success when the top-k retrieval setting is 5, matching white-box performance, and maintains a consistent advantage of approximately 5 percentage points across different top-k values. It also outperforms existing black-box baselines by 14.5 percentage points under various defense strategies. Furthermore, our method successfully compromises a commercial RAG system deployed on Alibaba's BaiLian platform, demonstrating its practical threat in real-world applications. These findings underscore the need for more robust and secure RAG frameworks to defend against poisoning attacks.

Figures (13)

• Figure 1: ASR performance under combined perplexity, duplication, paraphrasing, and knowledge expansion defenses.
• Figure 2: Comparison of Adversarial Texts across Different Approaches.
• Figure 3: Overview of the CPA-RAG poisoning attack on RAG systems.
• Figure 4: CPA-RAG adversarial text generation pipeline. (1) Information collection: specify the target question, answer, and supporting knowledge. (2) Text initialization: generate candidate poisons via prompt-based sampling across LLMs. (3) Iterative refinement: optimize texts with retriever feedback and multi-model guidance to enhance covertness and retrievability.
• Figure 5: Comparing readability, fluency, grammar errors, and repetition across CPA-RAG, PoisonedRAG(B), PoisonedRAG(W), and natural texts.