Empirical Assessment of the Code Comprehension Effort Needed to Attack Programs Protected with Obfuscation

TL;DR

The paper addresses the challenge of evaluating obfuscation potency by conducting a controlled experiment with 152 MSc students to measure how two Tigress obfuscation variants, alone and in a layered combination, impede code comprehension and extend task time. It demonstrates that layering protections significantly lowers the odds of successful understanding compared with single obfuscations, while time increases are modest but present, and attributes these effects to both obfuscation and underlying code complexity. Importantly, the study provides empirical evidence that objective code metrics (e.g., $KLoC$) correlate with attacker success, bridging a gap between theory and practice, and offers a replication package for broader adoption. The findings have practical implications for selecting protection strategies and for building predictive models of attacker effort, guiding defenders in pricing and implementing layered protections. The work also lays groundwork for future research involving additional protections, professional attackers, and larger, more diverse codebases, to refine potency models and their predictive power.

Abstract

Evaluating the effectiveness of software protection is crucial for selecting the most effective methods to safeguard assets within software applications. Obfuscation involves techniques that deliberately modify software to make it more challenging to understand and reverse-engineer, while maintaining its original functionality. Although obfuscation is widely adopted, its effectiveness remains largely unexplored and unthoroughly evaluated. This paper presents a controlled experiment involving Master's students performing code comprehension tasks on applications hardened with obfuscation. The experiment's goals are to assess the effectiveness of obfuscation in delaying code comprehension by attackers and to determine whether complexity metrics can accurately predict the impact of these protections on success rates and durations of code comprehension tasks. The study is the first to evaluate the effect of layering multiple obfuscation techniques on a single piece of protected code. It also provides experimental evidence of the correlation between objective metrics of the attacked code and the likelihood of a successful attack, bridging the gap between objective and subjective approaches to estimating potency. Finally, the paper highlights significant aspects that warrant additional analysis and opens new avenues for further experiments.

Figures (9)

• Figure 1: Score distribution of the C skill tests.
• Figure 2: Success rate by obfuscation technique and application (segments indicate the 95% CI).
• Figure 3: Code comprehension time by Treatment and Application
• Figure 4: Success rate vs. Code size
• Figure 5: Code comprehension time vs. Code size