HogVul: Black-box Adversarial Code Generation Framework Against LM-based Vulnerability Detectors

TL;DR

This work tackles the robustness evaluation of LM-based vulnerability detectors by introducing HogVul, a black-box adversarial code generation framework that jointly explores lexical and syntax perturbations. The method unifies these perturbations through a dual-channel PSO optimization with stagnation-aware switching and cross-channel information fusion, enabling efficient discovery of high-impact adversarial code while preserving functionality. Empirical results across multiple datasets and victim models show substantial attack efficacy and robustness, with HogVul achieving significantly higher ASR and favorable query efficiency metrics (APQ) than strong baselines. The findings highlight the importance of hybrid perturbation strategies for exposing vulnerabilities and underscore the need for defenses that address multi-level code representations in vulnerability detection systems.

Abstract

Recent advances in software vulnerability detection have been driven by Language Model (LM)-based approaches. However, these models remain vulnerable to adversarial attacks that exploit lexical and syntax perturbations, allowing critical flaws to evade detection. Existing black-box attacks on LM-based vulnerability detectors primarily rely on isolated perturbation strategies, limiting their ability to efficiently explore the adversarial code space for optimal perturbations. To bridge this gap, we propose HogVul, a black-box adversarial code generation framework that integrates both lexical and syntax perturbations under a unified dual-channel optimization strategy driven by Particle Swarm Optimization (PSO). By systematically coordinating two-level perturbations, HogVul effectively expands the search space for adversarial examples, enhancing the attack efficacy. Extensive experiments on four benchmark datasets demonstrate that HogVul achieves an average attack success rate improvement of 26.05\% over state-of-the-art baseline methods. These findings highlight the potential of hybrid optimization strategies in exposing model vulnerabilities.

Figures (9)

• Figure 1: A simple example of adversarial code misleading models.
• Figure 2: The overview of HogVul. It operates in four stages: ①Diversity-enhanced Initialization, ②PSO Driven Lexical Perturbation, ③Structure-Aware Code Transformation and ④Dual-Channel Cooperative Optimization.
• Figure 3: Visualization of particle trajectories in the code space during iterations of different optimization algorithms.
• Figure 4: The diversity of the population in the last iteration.
• Figure 5: Comparison of ASR across component settings.