Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AEGIS: White-Box Attack Path Generation using LLMs and Training Effectiveness Evaluation for Large-Scale Cyber Defence Exercises

Ivan K. Tung, Yu Xiang Shi, Alex Chien, Wenkai Liu, Lawrence Zheng

TL;DR

This work tackles the heavy burden of crafting cyber defence exercise (CDX) attack paths by introducing AEGIS, a white-box, LLM-guided system that uses Monte Carlo Tree Search over real exploit execution. It decomposes attack-path generation into reconnaissance, exploit search, empirical validation, and staged path search/execution, enabling errors to be caught early and reducing development time. An empirical evaluation at CIDeX 2025 shows AEGIS-generated paths achieve statistically equivalent training experience to human-authored paths across perceived learning, engagement, believability, and challenge, validating its utility as a substitute in training scenarios. The approach also highlights a practical defense research direction: automated, scalable scenario generation that preserves pedagogical value while shifting expert effort toward scenario design and validation of assets and risks.

Abstract

Creating attack paths for cyber defence exercises requires substantial expert effort. Existing automation requires vulnerability graphs or exploit sets curated in advance, limiting where it can be applied. We present AEGIS, a system that generates attack paths using LLMs, white-box access, and Monte Carlo Tree Search over real exploit execution. LLM-based search discovers exploits dynamically without pre-existing vulnerability graphs, while white-box access enables validating exploits in isolation before committing to attack paths. Evaluation at CIDeX 2025, a large-scale exercise spanning 46 IT hosts, showed that AEGIS-generated paths are comparable to human-authored scenarios across four dimensions of training experience (perceived learning, engagement, believability, challenge). Results were measured with a validated questionnaire extensible to general simulation-based training. By automating exploit chain discovery and validation, AEGIS reduces scenario development from months to days, shifting expert effort from technical validation to scenario design.

AEGIS: White-Box Attack Path Generation using LLMs and Training Effectiveness Evaluation for Large-Scale Cyber Defence Exercises

TL;DR

This work tackles the heavy burden of crafting cyber defence exercise (CDX) attack paths by introducing AEGIS, a white-box, LLM-guided system that uses Monte Carlo Tree Search over real exploit execution. It decomposes attack-path generation into reconnaissance, exploit search, empirical validation, and staged path search/execution, enabling errors to be caught early and reducing development time. An empirical evaluation at CIDeX 2025 shows AEGIS-generated paths achieve statistically equivalent training experience to human-authored paths across perceived learning, engagement, believability, and challenge, validating its utility as a substitute in training scenarios. The approach also highlights a practical defense research direction: automated, scalable scenario generation that preserves pedagogical value while shifting expert effort toward scenario design and validation of assets and risks.

Abstract

Creating attack paths for cyber defence exercises requires substantial expert effort. Existing automation requires vulnerability graphs or exploit sets curated in advance, limiting where it can be applied. We present AEGIS, a system that generates attack paths using LLMs, white-box access, and Monte Carlo Tree Search over real exploit execution. LLM-based search discovers exploits dynamically without pre-existing vulnerability graphs, while white-box access enables validating exploits in isolation before committing to attack paths. Evaluation at CIDeX 2025, a large-scale exercise spanning 46 IT hosts, showed that AEGIS-generated paths are comparable to human-authored scenarios across four dimensions of training experience (perceived learning, engagement, believability, challenge). Results were measured with a validated questionnaire extensible to general simulation-based training. By automating exploit chain discovery and validation, AEGIS reduces scenario development from months to days, shifting expert effort from technical validation to scenario design.
Paper Structure (51 sections, 1 equation, 8 figures, 10 tables)

This paper contains 51 sections, 1 equation, 8 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Automated Attack Planning
  4. LLM-Based Penetration Testing
  5. CDX Automation and Evaluation
  6. Exploratory Study
  7. Testing Strategy
  8. Findings
  9. Comparison with Expert Behaviour
  10. Design Motivation
  11. Methodology
  12. Overview
  13. Reconnaissance Scanner
  14. Exploit Searcher
  15. Exploit Validator
  16. ...and 36 more sections

Figures (8)

  • Figure 1: Network topology.
  • Figure 2: The AEGIS pipeline from reconnaissance to validated exploit sequences.
  • Figure 3: Terminal use agent used in Exploit Validator and Attack Path Simulator where all terminal outputs are aggregated into a single structured prompt.
  • Figure 4: MCTS attack path search phases.
  • Figure 5: An attack path AEGIS generated in the CIDeX 2025 network which compromises the domain controller. This is later used in AEGIS scenario B.
  • ...and 3 more figures