A Systematic Study of Code Obfuscation Against LLM-based Vulnerability Detection

TL;DR

This study examines how code obfuscation influences LLM-based vulnerability detection, addressing robustness concerns as LLMs are increasingly used in software auditing. It introduces a comprehensive taxonomy of 19 obfuscation techniques across layout, data-flow, and control-flow categories and evaluates them across Solidity, C, C++, and Python with 15 LLMs and two coding agents, using a fine-grained four-level scoring scheme. Key findings reveal a dual effect: obfuscation can degrade detection and, in some cases, unexpectedly improve it; robustness exhibits an 8B-parameter boundary and trade-offs between reasoning capability and generalization; dataset characteristics and vulnerability types strongly modulate outcomes. The work suggests directions for future research, including obfuscation-aware training, cross-layer semantic reasoning, and precision-preserving hot-plugging to build more robust, agent-based vulnerability detectors.

Abstract

As large language models (LLMs) are increasingly adopted for code vulnerability detection, their reliability and robustness across diverse vulnerability types have become a pressing concern. In traditional adversarial settings, code obfuscation has long been used as a general strategy to bypass auditing tools, preserving exploitability without tampering with the tools themselves. Numerous efforts have explored obfuscation methods and tools, yet their capabilities differ in terms of supported techniques, granularity, and programming languages, making it difficult to systematically assess their impact on LLM-based vulnerability detection. To address this gap, we provide a structured systematization of obfuscation techniques and evaluate them under a unified framework. Specifically, we categorize existing obfuscation methods into three major classes (layout, data flow, and control flow) covering 11 subcategories and 19 concrete techniques. We implement these techniques across four programming languages (Solidity, C, C++, and Python) using a consistent LLM-driven approach, and evaluate their effects on 15 LLMs spanning four model families (DeepSeek, OpenAI, Qwen, and LLaMA), as well as on two coding agents (GitHub Copilot and Codex). Our findings reveal both positive and negative impacts of code obfuscation on LLM-based vulnerability detection, highlighting conditions under which obfuscation leads to performance improvements or degradations. We further analyze these outcomes with respect to vulnerability characteristics, code properties, and model attributes. Finally, we outline several open problems and propose future directions to enhance the robustness of LLMs for real-world vulnerability detection.

Figures (11)

• Figure 1: Distribution of downgrade and upgrade samples on obfuscation technique.
• Figure 2: Distribution of downgrade and upgrade samples on model.
• Figure 3: Average Upgrade/Degrade per Model (by Series).
• Figure 4: Distribution of downgrade and upgrade samples on model size.
• Figure 5: Distribution of downgrade and upgrade samples on vulnerability type.