Exploring the Security Threats of Retriever Backdoors in Retrieval-Augmented Code Generation

TL;DR

This work identifies retriever backdoors in Retrieval-Augmented Code Generation as a practical threat, showing that tiny, stealthy poisonings can bias code retrieval toward vulnerable snippets and substantially increase vulnerable code generation. It introduces VenomRACG, a two-phase attack that first trains a backdoor-sensitive retriever via semantic disruption and targeted trigger design, then poisons a small set of vulnerable code snippets in the knowledge base. The method achieves high ASR@k and VR across white-box and black-box settings while maintaining normal performance on benign queries, and it largely evades current detectors. The findings underscore an urgent need for security measures in RACG supply chains and provide open-source artifacts to spur defense research and robust evaluation.

Abstract

Retrieval-Augmented Code Generation (RACG) is increasingly adopted to enhance Large Language Models for software development, yet its security implications remain dangerously underexplored. This paper conducts the first systematic exploration of a critical and stealthy threat: backdoor attacks targeting the retriever component, which represents a significant supply-chain vulnerability. It is infeasible to assess this threat realistically, as existing attack methods are either too ineffective to pose a real danger or are easily detected by state-of-the-art defense mechanisms spanning both latent-space analysis and token-level inspection, which achieve consistently high detection rates. To overcome this barrier and enable a realistic analysis, we first developed VenomRACG, a new class of potent and stealthy attack that serves as a vehicle for our investigation. Its design makes poisoned samples statistically indistinguishable from benign code, allowing the attack to consistently maintain low detectability across all evaluated defense mechanisms. Armed with this capability, our exploration reveals a severe vulnerability: by injecting vulnerable code equivalent to only 0.05% of the entire knowledge base size, an attacker can successfully manipulate the backdoored retriever to rank the vulnerable code in its top-5 results in 51.29% of cases. This translates to severe downstream harm, causing models like GPT-4o to generate vulnerable code in over 40% of targeted scenarios, while leaving the system's general performance intact. Our findings establish that retriever backdooring is not a theoretical concern but a practical threat to the software development ecosystem that current defenses are blind to, highlighting the urgent need for robust security measures.

Figures (3)

• Figure 1: Overview of VenomRACG.
• Figure 2: Comparison between White-box and Black-box Code Base across Four Feature Dimensions.
• Figure 3: Comparison of Vulnerability Detection Methods: LLM-as-a-Judge vs. Static Analysis vs. Human Review