Poisoned Source Code Detection in Code Models
Ehab Ghannoum, Mohammad Ghafari
TL;DR
This work tackles data poisoning in source-code processing by introducing CodeGarrison (CG), a hybrid detector that leverages multiple code embeddings to identify poisoned samples before model fine-tuning. CG is evaluated against the state-of-the-art ONION across four poisoning techniques (DAMP, MHM, ALTER, CodeFooler) and shows superior accuracy and robustness, including toward unseen attacks. The study analyzes feature contributions and demonstrates that embedding combinations, especially Code2Vec, CodeBERT, and FastText, yield the best performance, with CodeBERT offering strong class separation in embedding space. Practically, CG can be deployed as a preprocessing step or an IDE-time detector, enhancing the reliability of code-model outputs and supporting safer deployment of code-generation and analysis tools.
Abstract
Deep learning models have gained popularity for conducting various tasks involving source code. However, their black-box nature raises concerns about potential risks. One such risk is a poisoning attack, where an attacker intentionally contaminates the training set with malicious samples to mislead the model's predictions in specific scenarios. To protect source code models from poisoning attacks, we introduce CodeGarrison (CG), a hybrid deep-learning model that relies on code embeddings to identify poisoned code samples. We evaluated CG against the state-of-the-art technique ONION for detecting poisoned samples generated by DAMP, MHM, ALERT, as well as a novel poisoning technique named CodeFooler. Results showed that CG significantly outperformed ONION with an accuracy of 93.5%. We also tested CG's robustness against unknown attacks, achieving an average accuracy of 85.6% in identifying poisoned samples across the four attacks mentioned above.