EvadeDroid: A Practical Evasion Attack on Machine Learning for Black-box Android Malware Detection

TL;DR

EvadeDroid tackles the practicality gap in Android malware evasion by operating in a fully black-box, zero-knowledge setting and manipulating real APKs in the problem space. It builds a dynamic action set of gadgets from benign donors using n-gram opcode similarity and applies them with a query-efficient Random Search to produce real-world adversarial examples that mislead multiple detectors. The approach achieves high evasion rates across academic detectors (e.g., DREBIN, Sec-SVM, ADE-MA, MaMaDroid, Opcode-SVM) and shows substantial resilience against commercial antiviruses, while maintaining functionality and plausibility. These results demonstrate the feasibility of real-world adversarial testing and provide a foundation for developing robust defenses through adversarial training and detector hardening.

Abstract

Over the last decade, researchers have extensively explored the vulnerabilities of Android malware detectors to adversarial examples through the development of evasion attacks; however, the practicality of these attacks in real-world scenarios remains arguable. The majority of studies have assumed attackers know the details of the target classifiers used for malware detection, while in reality, malicious actors have limited access to the target classifiers. This paper introduces EvadeDroid, a problem-space adversarial attack designed to effectively evade black-box Android malware detectors in real-world scenarios. EvadeDroid constructs a collection of problem-space transformations derived from benign donors that share opcode-level similarity with malware apps by leveraging an n-gram-based approach. These transformations are then used to morph malware instances into benign ones via an iterative and incremental manipulation strategy. The proposed manipulation technique is a query-efficient optimization algorithm that can find and inject optimal sequences of transformations into malware apps. Our empirical evaluations, carried out on 1K malware apps, demonstrate the effectiveness of our approach in generating real-world adversarial examples in both soft- and hard-label settings. Our findings reveal that EvadeDroid can effectively deceive diverse malware detectors that utilize different features with various feature types. Specifically, EvadeDroid achieves evasion rates of 80%-95% against DREBIN, Sec-SVM, ADE-MA, MaMaDroid, and Opcode-SVM with only 1-9 queries. Furthermore, we show that the proposed problem-space adversarial attack is able to preserve its stealthiness against five popular commercial antiviruses with an average of 79% evasion rate, thus demonstrating its feasibility in the real world.

Figures (8)

• Figure 1: Overview of EvadeDroid's pipeline.
• Figure 2: The functionality of EvadeDroid in generating real-world adversarial malware apps. The dark red and dark green samples are, respectively, the inaccessible malware and benign samples that have been used for training the malware classifier. Light red and light green samples represent, respectively, accessible malware and benign samples in the wild. The blue and purple samples are manipulated malware apps and AEs, respectively.
• Figure 3: Applying a problem-space transformation (i.e., gadget) into a malware app involves injecting the gadget extracted from an API call entry point (e.g., SmsManager) in a donor into an obfuscated false condition statement within the malware app. The code snippets are displayed in Java representation to facilitate better understanding.
• Figure 4: The temporal distribution of training samples. The dataset b26 lacked clarity regarding the release dates of the $\approx 1.5K$ samples in our training set.
• Figure 5: ROC curves of DREBIN, Sec-SVM, ADE-MA, MaMaDroid, and Opcode-SVM in the absence of adversarial attacks. The regions with translucent colors that encompass the lines are standard deviations.