Network Intrusion Datasets: A Survey, Limitations, and Recommendations

TL;DR

The paper addresses the persistent challenge of data quality and accessibility in network intrusion detection by conducting the largest systematic review of public NIDS datasets to date. It collects 13 data-properties for 89 datasets, analyzes dataset popularity in recent top-tier research, and highlights domain-specific limitations such as timeliness, realism, and labeling ambiguities. The authors offer concrete recommendations for dataset selection, creation, and usage, and outline future directions for data generation, validation, and publishing to improve benchmarking robustness. Overall, the work aims to steer NIDS research toward higher-quality, more realistic data and reproducible, transparent evaluation practices with timely data updates.

Abstract

Data-driven cyberthreat detection has become a crucial defense technique in modern cybersecurity. Network defense, supported by Network Intrusion Detection Systems (NIDSs), has also increasingly adopted data-driven approaches, leading to greater reliance on data. Despite the importance of data, its scarcity has long been recognized as a major obstacle in NIDS research. In response, the community has published many new datasets recently. However, many of them remain largely unknown and unanalyzed, leaving researchers uncertain about their suitability for specific use cases. In this paper, we aim to address this knowledge gap by performing a systematic literature review (SLR) of 89 public datasets for NIDS research. Each dataset is comparatively analyzed across 13 key properties, and its potential applications are outlined. Beyond the review, we also discuss domain-specific challenges and common data limitations to facilitate a critical view on data quality. To aid in data selection, we conduct a dataset popularity analysis in contemporary state-of-the-art NIDS research. Furthermore, the paper presents best practices for dataset selection, generation, and usage. By providing a comprehensive overview of the domain and its data, this work aims to guide future research toward improving data quality and the robustness of NIDS solutions.

Figures (10)

• Figure 1: Summary of Network Intrusion Detection domain-specific properties affecting data collection, handling, and interpretation.
• Figure 2: Limitations of datasets for network intrusion detection. As illustrated, some are a direct consequence of domain-specific properties and are challenging to address (e.g., timeliness), some are primarily caused by a human factor and can be mitigated completely (e.g., documentation), and those that lay in an intersection of both can be addressed only partially (e.g., class imbalance -- NID traffic is naturally imbalanced, but data authors might attempt to reduce it).
• Figure 3: Taxonomy of datasets for Network Intrusion Detection based on their focus. Note that special-purpose categories are not exclusive, as a single dataset can both focus on a particular intrusion type captured in a specific environment.
• Figure 4: Analysis of the utilized datasets from the contemporary NIDS research. Most papers used custom datasets collected specifically for the given research (category Custom), whereas the majority of them were not publicly shared. The category Other (listed) represents datasets included in the survey (Table \ref{['tab:data_survey']}) but used only once, so they were grouped. The category Other (unlisted) groups datasets not included in the survey.
• Figure 5: The number of NID datasets by the year of publication. As shown, the number of released datasets has grown polynomially in the past years.