Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Comparison of Multiple Classifiers for Android Malware Detection with Emphasis on Feature Insights Using CICMalDroid 2020 Dataset

Md Min-Ha-Zul Abedin, Tazqia Mehrub

TL;DR

This paper tackles robust Android malware detection by leveraging CICMalDroid2020, containing 17,341 labeled apps across multiple malware families and Benign instances. It trains seven classifiers on a rich 564-dimensional hybrid feature vector that fuses static and dynamic signals, evaluated under three preprocessing schemes, and emphasizes interpretability via a surrogate decision tree. The results show gradient-boosting ensembles, especially XGBoost on original features, achieve top performance (~0.975 accuracy) with clear feature drivers such as package name, main activity, and target SDK, while dimensionality reduction via PCA degrades performance. The study provides high-fidelity baselines for deployment and suggests future work in semi-supervised or transformer-based models and ongoing dataset updates.

Abstract

Accurate Android malware detection was critical for protecting users at scale. Signature scanners lagged behind fast release cycles on public app stores. We aimed to build a trustworthy detector by pairing a comprehensive dataset with a rigorous, transparent evaluation, and to identify interpretable drivers of decisions. We used CICMalDroid2020, which contained 17,341 apps across Benign, Adware, Banking, SMS malware, and Riskware. We extracted 301 static and 263 dynamic features into a 564 dimensional hybrid vector, then evaluated seven classifiers under three schemes, original features, principal component analysis, PCA, and linear discriminant analysis, LDA, with a 70 percent training and 30 percent test split. Results showed that gradient boosting on the original features performed best. XGBoost achieved 0.9747 accuracy, 0.9703 precision, 0.9731 recall, and 0.9716 F1, and the confusion matrix indicated rare benign labels for malicious apps. HistGradientBoosting reached 0.9741 accuracy and 0.9708 F1, while CatBoost and Random Forest were slightly lower at 0.9678 and 0.9687 accuracy with 0.9636 and 0.9637 F1. KNN and SVM lagged. PCA reduced performance for all models, with XGBoost dropping to 0.9164 accuracy and 0.8988 F1. LDA maintained mid 90s accuracy and clarified separable clusters in projections. A depth two surrogate tree highlighted package name, main activity, and target SDK as key drivers. These findings established high fidelity supervised baselines for Android malware detection and indicated that rich hybrid features with gradient boosting offered a practical and interpretable foundation for deployment.

Comparison of Multiple Classifiers for Android Malware Detection with Emphasis on Feature Insights Using CICMalDroid 2020 Dataset

TL;DR

This paper tackles robust Android malware detection by leveraging CICMalDroid2020, containing 17,341 labeled apps across multiple malware families and Benign instances. It trains seven classifiers on a rich 564-dimensional hybrid feature vector that fuses static and dynamic signals, evaluated under three preprocessing schemes, and emphasizes interpretability via a surrogate decision tree. The results show gradient-boosting ensembles, especially XGBoost on original features, achieve top performance (~0.975 accuracy) with clear feature drivers such as package name, main activity, and target SDK, while dimensionality reduction via PCA degrades performance. The study provides high-fidelity baselines for deployment and suggests future work in semi-supervised or transformer-based models and ongoing dataset updates.

Abstract

Accurate Android malware detection was critical for protecting users at scale. Signature scanners lagged behind fast release cycles on public app stores. We aimed to build a trustworthy detector by pairing a comprehensive dataset with a rigorous, transparent evaluation, and to identify interpretable drivers of decisions. We used CICMalDroid2020, which contained 17,341 apps across Benign, Adware, Banking, SMS malware, and Riskware. We extracted 301 static and 263 dynamic features into a 564 dimensional hybrid vector, then evaluated seven classifiers under three schemes, original features, principal component analysis, PCA, and linear discriminant analysis, LDA, with a 70 percent training and 30 percent test split. Results showed that gradient boosting on the original features performed best. XGBoost achieved 0.9747 accuracy, 0.9703 precision, 0.9731 recall, and 0.9716 F1, and the confusion matrix indicated rare benign labels for malicious apps. HistGradientBoosting reached 0.9741 accuracy and 0.9708 F1, while CatBoost and Random Forest were slightly lower at 0.9678 and 0.9687 accuracy with 0.9636 and 0.9637 F1. KNN and SVM lagged. PCA reduced performance for all models, with XGBoost dropping to 0.9164 accuracy and 0.8988 F1. LDA maintained mid 90s accuracy and clarified separable clusters in projections. A depth two surrogate tree highlighted package name, main activity, and target SDK as key drivers. These findings established high fidelity supervised baselines for Android malware detection and indicated that rich hybrid features with gradient boosting offered a practical and interpretable foundation for deployment.
Paper Structure (9 sections, 9 figures, 2 tables)

This paper contains 9 sections, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Dataset Description
  3. Methodology
  4. Results and Discussion
  5. Effect of preprocessing
  6. Overall precision comparisons
  7. Per-class analysis
  8. Model interpretability
  9. Conclusion

Figures (9)

  • Figure 1: Illustrates the balanced distribution of benign and sub-class of malware samples.
  • Figure 2: Presents the LDA projection of the dataset, demonstrating moderate separability.
  • Figure 3: Depicts the top ten important features in the dataset
  • Figure 4: Depicts the t-SNE visualization, showcasing significantly improved separation between malware sub-class and benign classes due to nonlinear dimensionality reduction properties.
  • Figure 5: Performance of different model with feature set after applying PCA.
  • ...and 4 more figures