A Critical Assessment of Interpretable and Explainable Machine Learning for Intrusion Detection

TL;DR

This paper critically assesses feature-based interpretable and explainable ML for intrusion detection, revealing that many datasets are imbalanced and that explanations are often inconsistent across models, explanations, and training settings. It demonstrates that simple, interpretable models such as Decision Trees achieve near-perfect performance on common intrusion datasets, challenging the value of opaque models like DNNs in this context. By introducing cross-explanations and analyzing learning-process constituents and feature correlations, the authors show that explanations are frequently unstable and not transferable, thereby limiting their practical utility. The work advocates shifting away from feature-importance explanations toward prototype-based or counterfactual approaches and highlights the need for proper metrics ($BA$, $MCC$) to accurately assess model performance in cybersecurity tasks. Overall, the study provides guidance for more reliable evaluation and actionable explanation strategies in intrusion detection systems.

Abstract

There has been a large number of studies in interpretable and explainable ML for cybersecurity, in particular, for intrusion detection. Many of these studies have significant amount of overlapping and repeated evaluations and analysis. At the same time, these studies overlook crucial model, data, learning process, and utility related issues and many times completely disregard them. These issues include the use of overly complex and opaque ML models, unaccounted data imbalances and correlated features, inconsistent influential features across different explanation methods, the inconsistencies stemming from the constituents of a learning process, and the implausible utility of explanations. In this work, we empirically demonstrate these issues, analyze them and propose practical solutions in the context of feature-based model explanations. Specifically, we advise avoiding complex opaque models such as Deep Neural Networks and instead using interpretable ML models such as Decision Trees as the available intrusion datasets are not difficult for such interpretable models to classify successfully. Then, we bring attention to the binary classification metrics such as Matthews Correlation Coefficient (which are well-suited for imbalanced datasets. Moreover, we find that feature-based model explanations are most often inconsistent across different settings. In this respect, to further gauge the extent of inconsistencies, we introduce the notion of cross explanations which corroborates that the features that are determined to be impactful by one explanation method most often differ from those by another method. Furthermore, we show that strongly correlated data features and the constituents of a learning process, such as hyper-parameters and the optimization routine, become yet another source of inconsistent explanations. Finally, we discuss the utility of feature-based explanations.

Figures (12)

• Figure 1: Interpretability vs. flexibility of ML models. Note that the relationship shown here is a qualitative overview of ML models.
• Figure 2: The plots of models $M_{step}$ and $M_{smooth}$ where $c_1 = 0.9$, $c_2 = 0.1$, and $threshold = 7$. The plot is inspired by Chapter 33 - Section 33.1.2 of pml2Book.
• Figure 3: The top features based on the FIs, PIs, and SHAPs for DTs for the 5G dataset.
• Figure 4: The top features based on the FIs, PIs, and SHAPs for DTs for the UDBLag dataset with two different random seeds.
• Figure 5: The resulting DTs for the two datasets.