Automated Post-Incident Policy Gap Analysis via Threat-Informed Evidence Mapping using Large Language Models
Huan Lin Oh, Jay Yong Jun Jie, Mandy Lee Ling Siu, Jonathan Pan
TL;DR
Post-incident reviews are essential but labor-intensive; the paper asks whether LLMs can autonomously analyze system evidence and identify policy gaps. The authors present an end-to-end agentic workflow that ingests logs, maps observed behaviours to MITRE ATT&CK, and evaluates policy adequacy with traceability. Using a simulated brute-force attack against Windows OpenSSH (T1110), the approach demonstrates that LLMs can interpret log-derived evidence and produce remediation with evidence-to-policy links. The results indicate potential gains in efficiency, consistency, and auditability, while noting that human oversight remains crucial in high-stakes cybersecurity decisions.
Abstract
Cybersecurity post-incident reviews are essential for identifying control failures and improving organisational resilience, yet they remain labour-intensive, time-consuming, and heavily reliant on expert judgment. This paper investigates whether Large Language Models (LLMs) can augment post-incident review workflows by autonomously analysing system evidence and identifying security policy gaps. We present a threat-informed, agentic framework that ingests log data, maps observed behaviours to the MITRE ATT&CK framework, and evaluates organisational security policies for adequacy and compliance. Using a simulated brute-force attack scenario against a Windows OpenSSH service (MITRE ATT&CK T1110), the system leverages GPT-4o for reasoning, LangGraph for multi-agent workflow orchestration, and LlamaIndex for traceable policy retrieval. Experimental results indicate that the LLM-based pipeline can interpret log-derived evidence, identify insufficient or missing policy controls, and generate actionable remediation recommendations with explicit evidence-to-policy traceability. Unlike prior work that treats log analysis and policy validation as isolated tasks, this study integrates both into a unified end-to-end proof-of-concept post-incident review framework. The findings suggest that LLM-assisted analysis has the potential to improve the efficiency, consistency, and auditability of post-incident evaluations, while highlighting the continued need for human oversight in high-stakes cybersecurity decision-making.