SoK: Adversarial Evasion Attacks Practicality in NIDS Domain and the Impact of Dynamic Learning

TL;DR

This SoK assesses the practicality of adversarial evasion attacks against ML-based NIDS by constructing an attack-tree threat model and a taxonomy of practicality issues, then evaluating how dynamic learning (continuous retraining) affects attack viability. The authors synthesize literature from 2022 onward, identify leaves with questionable feasibility, and review several practically motivated attacks, including random perturbations, side-channel querying, and surrogate-model approaches. Through experiments on the CSE-CIC-IDS2018 dataset with multiple NIDS architectures, they show that continuous retraining can significantly mitigate attack effectiveness, though results vary by model and attack type, suggesting that real-world defenses must consider dynamic deployment. Overall, the work highlights a substantial gap between existing research and real-world practicality, stresses the importance of threat-model realism, and calls for more study on dynamic defenses and robust, diverse network-security datasets.

Abstract

Machine Learning (ML) has become pervasive, and its deployment in Network Intrusion Detection Systems (NIDS) is inevitable due to its automated nature and high accuracy compared to traditional models in processing and classifying large volumes of data. However, ML has been found to have several flaws, most importantly, adversarial attacks, which aim to trick ML models into producing faulty predictions. While most adversarial attack research focuses on computer vision datasets, recent studies have explored the suitability of these attacks against ML-based network security entities, especially NIDS, due to the wide difference between different domains regarding the generation of adversarial attacks. To further explore the practicality of adversarial attacks against ML-based NIDS in-depth, this paper presents several key contributions: identifying numerous practicality issues for evasion adversarial attacks on ML-NIDS using an attack tree threat model, introducing a taxonomy of practicality issues associated with adversarial attacks against ML-based NIDS, identifying specific leaf nodes in our attack tree that demonstrate some practicality for real-world implementation and conducting a comprehensive review and exploration of these potentially viable attack approaches, and investigating how the dynamicity of real-world ML models affects evasion adversarial attacks against NIDS. Our experiments indicate that continuous re-training, even without adversarial training, can reduce the effectiveness of adversarial attacks. While adversarial attacks can compromise ML-based NIDSs, our aim is to highlight the significant gap between research and real-world practicality in this domain, which warrants attention.

Figures (18)

• Figure 1: Systematic Literature Review
• Figure 2: The Deployment of Network Intrusion Detection System
• Figure 3: Attackers' Knowledge Types
• Figure 4: Feature-Space vs. Problem-Space Perturbations in NIDS Domain: If modifications occur at to the input network flow, this constitutes a problem-space attack; if perturbations occur at to the input feature vector, this constitutes a feature-space attack. In problem-space attacks, modifications such as increasing the size of one packet (the red one, for example) to alter the Max Packet Length feature must be mapped to the ML model's input feature vector. This process is called inverse feature mapping pierazzi2020intriguing. Note that the perturbations must remain malicious after both feature extraction and pre-processing (red perturbations), not just after feature extraction alone (purple perturbations).
• Figure 5: Defenses Against Adversarial Attacks