NatGVD: Natural Adversarial Example Attack towards Graph-based Vulnerability Detection
Avilash Rath, Weiliang Qi, Youpeng Li, Xinda Wang
TL;DR
NatGVD addresses the robustness gap of graph-based vulnerability detectors by introducing semantics-preserving, natural-looking code transformations that alter code-graph structure without changing behavior. The approach targets GNNs and graph-aware transformers, achieving notable evasion rates (up to 53.04%) while remaining compilable and humanly natural. The work also investigates defenses via ensemble methods and adversarial training, showing reduced evasion at the cost of more false positives or increased training data. Overall, NatGVD highlights the need for robust graph-based VD models and provides a practical framework for evaluating and strengthening them in real-world software security workflows.
Abstract
Graph-based models learn rich code graph structural information and present superior performance on various code analysis tasks. However, the robustness of these models against adversarial example attacks in the context of vulnerability detection remains an open question. This paper proposes NatGVD, a novel attack methodology that generates natural adversarial vulnerable code to circumvent GNN-based and graph-aware transformer-based vulnerability detectors. NatGVD employs a set of code transformations that modify graph structure while preserving code semantics. Instead of injecting dead or unrelated code like previous works, NatGVD considers naturalness requirements: generated examples should not be easily recognized by humans or program analysis tools. With extensive evaluation of NatGVD on state-of-the-art vulnerability detection systems, the results reveal up to 53.04% evasion rate across GNN-based detectors and graph-aware transformer-based detectors. We also explore potential defense strategies to enhance the robustness of these systems against NatGVD.