Simplicity by Obfuscation: Evaluating LLM-Driven Code Transformation with Semantic Elasticity

TL;DR

The paper addresses how to obfuscate Python code using large language models while preserving functionality. It introduces Semantic Elasticity, a metric that combines structural change, correctness, and code size to evaluate obfuscation quality, and benchmarks three LLMs (Claude-3.5-Sonnet, Gemini-1.5, GPT-4-Turbo) on 30 functions across five domains under zero- and few-shot prompts. Key findings show GPT-4-Turbo with few-shot prompting achieving the highest pass rate (~$P=0.8093$), while all models exhibit a surprising trend of reducing cyclomatic complexity, challenging traditional obfuscation assumptions; SE captures this trade-off and varies with prompting. The study provides a methodological framework for AI-driven obfuscation evaluation and highlights directions for security-focused research, including extending to additional languages and refining prompt strategies.

Abstract

Code obfuscation is the conversion of original source code into a functionally equivalent but less readable form, aiming to prevent reverse engineering and intellectual property theft. This is a challenging task since it is crucial to maintain functional correctness of the code while substantially disguising the input code. The recent development of large language models (LLMs) paves the way for practical applications in different domains, including software engineering. This work performs an empirical study on the ability of LLMs to obfuscate Python source code and introduces a metric (i.e., semantic elasticity) to measure the quality degree of obfuscated code. We experimented with 3 leading LLMs, i.e., Claude-3.5-Sonnet, Gemini-1.5, GPT-4-Turbo across 30 Python functions from diverse computational domains. Our findings reveal GPT-4-Turbo's remarkable effectiveness with few-shot prompting (81% pass rate versus 29% standard prompting), significantly outperforming both Gemini-1.5 (39%) and Claude-3.5-Sonnet (30%). Notably, we discovered a counter-intuitive "obfuscation by simplification" phenomenon where models consistently reduce rather than increase cyclomatic complexity. This study provides a methodological framework for evaluating AI-driven obfuscation while highlighting promising directions for leveraging LLMs in software security.

Figures (4)

• Figure 1: Proposed methodology.
• Figure 2: Distribution of obfuscation metrics across models.
• Figure 3: Correlation analysis.
• Figure 4: Complexity reduction analysis.