ADMIT: Few-shot Knowledge Poisoning Attacks on RAG-based Fact Checking

TL;DR

This work targets the robustness of RAG-based fact-checking by introducing ADMIT, a few-shot adversarial knowledge-poisoning framework that uses proxy verifiers and proxy passages to flip verification outcomes even when credible evidence coexists with adversarial content. ADMIT supports single-turn and multi-turn generation, employs search- or LLM-based proxy construction, and augments adversarial passages with retrieval prefixes to increase recall. Empirical results across 4 BEIR-derived datasets, 11 LLM verifiers, and 4 retrievers show ADMIT achieving an average ASR of $86\%$ at a poisoning rate of $0.93 \times 10^{-6}$, outperforming prior attacks by ~$11.2\%$ and transferring robustly to unseen models. Defenses such as fake-news detection, knowledge consolidation, and retrieval- or generation-time defenses offer partial mitigation, underscoring the need for provenance tracking and uncertainty-aware reasoning in RAG pipelines. Overall, the study reveals substantive vulnerabilities in real-world RAG-based fact-checking systems and motivates stronger safeguards for knowledge-grounded AI systems.

Abstract

Knowledge poisoning poses a critical threat to Retrieval-Augmented Generation (RAG) systems by injecting adversarial content into knowledge bases, tricking Large Language Models (LLMs) into producing attacker-controlled outputs grounded in manipulated context. Prior work highlights LLMs' susceptibility to misleading or malicious retrieved content. However, real-world fact-checking scenarios are more challenging, as credible evidence typically dominates the retrieval pool. To investigate this problem, we extend knowledge poisoning to the fact-checking setting, where retrieved context includes authentic supporting or refuting evidence. We propose \textbf{ADMIT} (\textbf{AD}versarial \textbf{M}ulti-\textbf{I}njection \textbf{T}echnique), a few-shot, semantically aligned poisoning attack that flips fact-checking decisions and induces deceptive justifications, all without access to the target LLMs, retrievers, or token-level control. Extensive experiments show that ADMIT transfers effectively across 4 retrievers, 11 LLMs, and 4 cross-domain benchmarks, achieving an average attack success rate (ASR) of 86\% at an extremely low poisoning rate of $0.93 \times 10^{-6}$, and remaining robust even in the presence of strong counter-evidence. Compared with prior state-of-the-art attacks, ADMIT improves ASR by 11.2\% across all settings, exposing significant vulnerabilities in real-world RAG-based fact-checking systems.

Figures (13)

• Figure 1: Comparison of attack strategies against RAG. Attacker injects malicious text (i.e., passage) into the knowledge database, then RAG answers the user query based on retrieved content from the poisoned knowledge database. Prompt Injection Attacks insert malicious instructions directly into the prompt to manipulate a predefined verdict, but often fail to induce coherent or plausible explanations from the model. General Knowledge Poisoning injects multiple malicious passages into the corpus; however, this method is easily mitigated if clean passages (i.e., gold evidence) are retrieved during inference. Few-Shot Knowledge Poisoning (Ours) introduces a single, minimal passage that not only overrides evidence but also misleads the LLM into producing the attacker’s target verdict along with a deceptive and contextually plausible justification.
• Figure 2: ASRs of Baseline methods ($k{=}5$) against 11 verifiers across 1–5 shot settings on four datasets, showing strong transferability to unseen LLMs. Full results, including recall, are in Appendix \ref{['Appendix_addtional_result']}, Table \ref{['tab:asr-arr-k=5']}, and Table \ref{['tab:asr-arr-k=10']}.
• Figure 3: Distribution of optimization iterations required by Admit. Most claims succeed within 1--3 iterations. The $y$-axis is on a log scale to reflect the heavy skew: a few extreme cases run to the maximum iteration limit.
• Figure 4: Fake News Detection
• Figure 5: ROC curves for PPL-based detection under statistical consistency defenses.