Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generation

TL;DR

This paper tackles responsibility attribution for poisoned knowledge in Retrieval-Augmented Generation (RAG) systems by introducing RAGOrigin, a black-box framework that localizes poisoned texts responsible for misgeneration. It combines adaptive attribution scope with three signals—embedding similarity, semantic correlation, and generation influence—into a unified responsibility score RS(u) and uses clustering to separate poisoned from benign texts within a narrowed scope. Across seven datasets and fifteen poisoning attacks, including adaptive strategies, RAGOrigin consistently achieves high detection accuracy with low false positives/negatives and demonstrates robustness to dynamic, noisy, and paraphrased conditions, while maintaining minimal overhead. The work advances practical forensics in RAG pipelines, enabling auditing, data-pipeline tracing, and targeted remediation in real-world deployments, with code provided for reproducibility and extension to broader scenarios.

Abstract

Retrieval-Augmented Generation (RAG) integrates external knowledge into large language models to improve response quality. However, recent work has shown that RAG systems are highly vulnerable to poisoning attacks, where malicious texts are inserted into the knowledge database to influence model outputs. While several defenses have been proposed, they are often circumvented by more adaptive or sophisticated attacks. This paper presents RAGOrigin, a black-box responsibility attribution framework designed to identify which texts in the knowledge database are responsible for misleading or incorrect generations. Our method constructs a focused attribution scope tailored to each misgeneration event and assigns a responsibility score to each candidate text by evaluating its retrieval ranking, semantic relevance, and influence on the generated response. The system then isolates poisoned texts using an unsupervised clustering method. We evaluate RAGOrigin across seven datasets and fifteen poisoning attacks, including newly developed adaptive poisoning strategies and multi-attacker scenarios. Our approach outperforms existing baselines in identifying poisoned content and remains robust under dynamic and noisy conditions. These results suggest that RAGOrigin provides a practical and effective solution for tracing the origins of corrupted knowledge in RAG systems. Our code is available at: https://github.com/zhangbl6618/RAG-Responsibility-Attribution

Figures (4)

• Figure 1: An illustrative example of a responsibility attribution system for RAG. (a) A user queries the RAG system, the RAG generates a misgeneration response; (b) responsibility attribution system traces the responsible poisoned texts based on the misgeneration event reported by the user.
• Figure 2: Illustration of attribution scope narrowing method.
• Figure 3: Results of RAGOrigin and baselines with different top-$K$ of RAG on NQ dataset.
• Figure 4: Results of RAGOrigin and baselines with different number ($M$) of poisoned texts on NQ dataset.