NeuroGenPoisoning: Neuron-Guided Attacks on Retrieval-Augmented Generation of LLM via Genetic Optimization of External Knowledge

TL;DR

NeuroGenPoisoning addresses vulnerabilities in Retrieval-Augmented Generation by coupling internal neuron attribution with a genetic algorithm to evolve poisoned external knowledge. It identifies Poison-Responsive Neurons via Integrated Gradients and optimizes external passages to maximize their activation, enabling high Population Overwrite Success Rate ($POSR$) across multiple datasets and open-source LLMs while preserving fluency. The approach scales to large adversarial pools and demonstrates effectiveness in knowledge-conflict scenarios, highlighting concrete security implications for RAG systems. The work also outlines directions for defenses that account for neuron-level sensitivities and potential black-box extensions.

Abstract

Retrieval-Augmented Generation (RAG) empowers Large Language Models (LLMs) to dynamically integrate external knowledge during inference, improving their factual accuracy and adaptability. However, adversaries can inject poisoned external knowledge to override the model's internal memory. While existing attacks iteratively manipulate retrieval content or prompt structure of RAG, they largely ignore the model's internal representation dynamics and neuron-level sensitivities. The underlying mechanism of RAG poisoning has not been fully studied and the effect of knowledge conflict with strong parametric knowledge in RAG is not considered. In this work, we propose NeuroGenPoisoning, a novel attack framework that generates adversarial external knowledge in RAG guided by LLM internal neuron attribution and genetic optimization. Our method first identifies a set of Poison-Responsive Neurons whose activation strongly correlates with contextual poisoning knowledge. We then employ a genetic algorithm to evolve adversarial passages that maximally activate these neurons. Crucially, our framework enables massive-scale generation of effective poisoned RAG knowledge by identifying and reusing promising but initially unsuccessful external knowledge variants via observed attribution signals. At the same time, Poison-Responsive Neurons guided poisoning can effectively resolves knowledge conflict. Experimental results across models and datasets demonstrate consistently achieving high Population Overwrite Success Rate (POSR) of over 90% while preserving fluency. Empirical evidence shows that our method effectively resolves knowledge conflict.

Figures (6)

• Figure 1: An overview of NeuroGenPoisoning. This attack framework generates adversarial external knowledge in RAG guided by LLM internal neuron attribution and genetic optimization. Considering the internal-external knowledge conflict, it enables the generation of a large pool of effective poisoned knowledge at scale.
• Figure 2: POSR Across Models and Datasets of NeuroGenPoisoning
• Figure 3: Log-scaled Poison-Responsiveness Score (Log(PRS)) across generations for SQuAD 2.0, TriviaQA, and WikiQA for different LLMs.
• Figure 4: POSR comparison between GA-Poison-Responsiveness Score (PRS) and GA-Similarity (Sim)
• Figure 5: Distribution of POSR across generations. Even though early generations have a wide spread with many low-POSR samples, our method steadily resolves knowledge conflicts and brings most queries to high POSR.