Constructing Multi-label Hierarchical Classification Models for MITRE ATT&CK Text Tagging
Andrew Crossman, Jonah Dodd, Viralam Ramamurthy Chaithanya Kumar, Riyaz Mohammed, Andrew R. Plummer, Chandra Sekharudu, Deepak Warrier, Mohammad Yekrangian
TL;DR
This work addresses automating MITRE ATT&CK text tagging by formalizing it as $\textsc{att\&ck}: D \mapsto T$, where $D$ is a document and $T$ encodes tactics and techniques. It proposes a stratified task-space and a bottom-up tagging pipeline using TF-IDF vectorization with an optional hashing step for security, and a stochastic gradient descent ($SGD$) classifier to produce outputs. Compared with GPT-4o, the SGD-based approach yields higher tactic accuracy (about 94%) and technique accuracy (about 82%), while operating without LLMs or retrieval-augmented components and enabling reuse on threat-scenario data; a public release of the MITRE_models toolkit supports practical adoption. The work demonstrates practical impact for security analysts, particularly in low-resource settings, and provides a bridge between cybersecurity practice and formal AIML methodology through explicit task-space formalization and reusable artifacts.
Abstract
MITRE ATT&CK is a cybersecurity knowledge base that organizes threat actor and cyber-attack information into a set of tactics describing the reasons and goals threat actors have for carrying out attacks, with each tactic having a set of techniques that describe the potential methods used in these attacks. One major application of ATT&CK is the use of its tactic and technique hierarchy by security specialists as a framework for annotating cyber-threat intelligence reports, vulnerability descriptions, threat scenarios, inter alia, to facilitate downstream analyses. To date, the tagging process is still largely done manually. In this technical note, we provide a stratified "task space" characterization of the MITRE ATT&CK text tagging task for organizing previous efforts toward automation using AIML methods, while also clarifying pathways for constructing new methods. To illustrate one of the pathways, we use the task space strata to stage-wise construct our own multi-label hierarchical classification models for the text tagging task via experimentation over general cyber-threat intelligence text -- using shareable computational tools and publicly releasing the models to the security community (via https://github.com/jpmorganchase/MITRE_models). Our multi-label hierarchical approach yields accuracy scores of roughly 94% at the tactic level, as well as accuracy scores of roughly 82% at the technique level. The models also meet or surpass state-of-the-art performance while relying only on classical machine learning methods -- removing any dependence on LLMs, RAG, agents, or more complex hierarchical approaches. Moreover, we show that GPT-4o model performance at the tactic level is significantly lower (roughly 60% accuracy) than our own approach. We also extend our baseline model to a corpus of threat scenarios for financial applications produced by subject matter experts.