Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Knowledge-to-Data: LLM-Driven Synthesis of Structured Network Traffic for Testbed-Free IDS Evaluation

Konstantinos E. Kampourakis, Vyron Kampourakis, Efstratios Chatzoglou, Georgios Kambourakis, Stefanos Gritzalis

TL;DR

This paper tackles the challenge of obtaining realistic, labeled IDS traffic by using Large Language Models as knowledge-to-data engines conditioned on protocol documentation, attack semantics, and explicit statistical rules, without fine-tuning or access to raw samples. It validates the AWID3 Wi-Fi intrusion benchmark through a three-stage pipeline—knowledge extraction, synthetic data generation in two phases, and a multi-faceted validation framework combining global, feature-level, structural, and cross-domain assessments. Results show that LLM-generated traffic can closely replicate real dataset properties, enabling high-performing gradient-boosting classifiers trained on synthetic data to generalize to real samples (e.g., $F1$ up to $0.956$ and cross-domain accuracy around $0.995$ with LightGBM). The work demonstrates a practical, testbed-free path for IDS experimentation, with strong implications for privacy-preserving, on-demand cybersecurity research and potential extension to new protocols and emerging threats.

Abstract

Realistic, large-scale, and well-labeled cybersecurity datasets are essential for training and evaluating Intrusion Detection Systems (IDS). However, they remain difficult to obtain due to privacy constraints, data sensitivity, and the cost of building controlled collection environments such as testbeds and cyber ranges. This paper investigates whether Large Language Models (LLMs) can operate as controlled knowledge-to-data engines for generating structured synthetic network traffic datasets suitable for IDS research. We propose a methodology that combines protocol documentation, attack semantics, and explicit statistical rules to condition LLMs without fine-tuning or access to raw samples. Using the AWID3 IEEE~802.11 benchmark as a demanding case study, we generate labeled datasets with four state-of-the-art LLMs and assess fidelity through a multi-level validation framework including global similarity metrics, per-feature distribution testing, structural comparison, and cross-domain classification. Results show that, under explicit constraints, LLM-generated datasets can closely approximate the statistical and structural characteristics of real network traffic, enabling gradient-boosting classifiers to achieve F1-scores up to 0.956 when evaluated on real samples. Overall, the findings suggest that constrained LLM-driven generation can facilitate on-demand IDS experimentation, providing a testbed-free, privacy-preserving alternative that overcomes the traditional bottlenecks of physical traffic collection and manual labeling.

Knowledge-to-Data: LLM-Driven Synthesis of Structured Network Traffic for Testbed-Free IDS Evaluation

TL;DR

This paper tackles the challenge of obtaining realistic, labeled IDS traffic by using Large Language Models as knowledge-to-data engines conditioned on protocol documentation, attack semantics, and explicit statistical rules, without fine-tuning or access to raw samples. It validates the AWID3 Wi-Fi intrusion benchmark through a three-stage pipeline—knowledge extraction, synthetic data generation in two phases, and a multi-faceted validation framework combining global, feature-level, structural, and cross-domain assessments. Results show that LLM-generated traffic can closely replicate real dataset properties, enabling high-performing gradient-boosting classifiers trained on synthetic data to generalize to real samples (e.g., up to and cross-domain accuracy around with LightGBM). The work demonstrates a practical, testbed-free path for IDS experimentation, with strong implications for privacy-preserving, on-demand cybersecurity research and potential extension to new protocols and emerging threats.

Abstract

Realistic, large-scale, and well-labeled cybersecurity datasets are essential for training and evaluating Intrusion Detection Systems (IDS). However, they remain difficult to obtain due to privacy constraints, data sensitivity, and the cost of building controlled collection environments such as testbeds and cyber ranges. This paper investigates whether Large Language Models (LLMs) can operate as controlled knowledge-to-data engines for generating structured synthetic network traffic datasets suitable for IDS research. We propose a methodology that combines protocol documentation, attack semantics, and explicit statistical rules to condition LLMs without fine-tuning or access to raw samples. Using the AWID3 IEEE~802.11 benchmark as a demanding case study, we generate labeled datasets with four state-of-the-art LLMs and assess fidelity through a multi-level validation framework including global similarity metrics, per-feature distribution testing, structural comparison, and cross-domain classification. Results show that, under explicit constraints, LLM-generated datasets can closely approximate the statistical and structural characteristics of real network traffic, enabling gradient-boosting classifiers to achieve F1-scores up to 0.956 when evaluated on real samples. Overall, the findings suggest that constrained LLM-driven generation can facilitate on-demand IDS experimentation, providing a testbed-free, privacy-preserving alternative that overcomes the traditional bottlenecks of physical traffic collection and manual labeling.
Paper Structure (16 sections, 4 equations, 3 figures, 3 tables)

This paper contains 16 sections, 4 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Knowledge Extraction
  4. Synthetic Dataset Generation
  5. Validation
  6. Results
  7. Strengths, Limitations, & Future Work
  8. Strengths
  9. Limitations
  10. Future Work
  11. Related Work
  12. Conclusions
  13. Confusion Matrix
  14. Ruleset Outline
  15. Acknowledgments.
  16. ...and 1 more sections

Figures (3)

  • Figure 1: The proposed three-stage testbed-free generation pipeline.
  • Figure 2: Two-dimensional PCA projections comparing real (orange) and LLM-generated (blue) AWID3 data across all four models.
  • Figure 3: Confusion matrices for the best-performing classifier (LightGBM) across all four LLM-generated datasets. All models demonstrate high true positive rates for both normal and attack classes.