Signature in Code Backdoor Detection, how far are we?
Quoc Hung Le, Thanh Le-Cong, Bach Le, Bowen Xu
TL;DR
The paper re-evaluates Spectral Signature defenses for backdoor detection in code models, revealing that commonly used default configurations are often suboptimal across modern code attacks. By exhaustively exploring a 28-configuration SS space on two pretrained code models (CodeBERT and CodeT5) and three attack types (Fixed, Grammatical, Adaptive), it demonstrates substantial ASR-D improvements (average ~41.7%) with better configurations. It also shows that the proxy metric NPV provides a stronger, cheaper surrogate for actual defense performance than Recall, enabling more practical defense tuning. The work offers actionable guidelines for configuring SS in code backdoor scenarios and highlights directions for future research in code-specific representations and broader task-model settings to bolster defense effectiveness.
Abstract
As Large Language Models (LLMs) become increasingly integrated into software development workflows, they also become prime targets for adversarial attacks. Among these, backdoor attacks are a significant threat, allowing attackers to manipulate model outputs through hidden triggers embedded in training data. Detecting such backdoors remains a challenge, and one promising approach is the use of Spectral Signature defense methods that identify poisoned data by analyzing feature representations through eigenvectors. While some prior works have explored Spectral Signatures for backdoor detection in neural networks, recent studies suggest that these methods may not be optimally effective for code models. In this paper, we revisit the applicability of Spectral Signature-based defenses in the context of backdoor attacks on code models. We systematically evaluate their effectiveness under various attack scenarios and defense configurations, analyzing their strengths and limitations. We found that the widely used setting of Spectral Signature in code backdoor detection is often suboptimal. Hence, we explored the impact of different settings of the key factors. We discovered a new proxy metric that can more accurately estimate the actual performance of Spectral Signature without model retraining after the defense.