Rule-ATT&CK Mapper (RAM): Mapping SIEM Rules to TTPs Using LLMs

TL;DR

Rule-ATT&CK Mapper (RAM) tackles the challenge of mapping structured SIEM rules to MITRE ATT&CK techniques without requiring model pretraining or fine tuning. It uses a multi stage AI agent pipeline inspired by prompt chaining to translate rules into natural language, enrich context via external sources, and map to ATT&CK techniques with transparent reasoning. Across Splunk based data, RAM demonstrates improved recall and precision when incorporating external contextual knowledge, with GPT-4-Turbo delivering the strongest performance among tested models. The work highlights RAM as a practical step toward automating threat detection workflows and guiding future integration of LLMs into cybersecurity operations.

Abstract

The growing frequency of cyberattacks has heightened the demand for accurate and efficient threat detection systems. SIEM platforms are important for analyzing log data and detecting adversarial activities through rule-based queries, also known as SIEM rules. The efficiency of the threat analysis process relies heavily on mapping these SIEM rules to the relevant attack techniques in the MITRE ATT&CK framework. Inaccurate annotation of SIEM rules can result in the misinterpretation of attacks, increasing the likelihood that threats will be overlooked. Existing solutions for annotating SIEM rules with MITRE ATT&CK technique labels have notable limitations: manual annotation of SIEM rules is both time-consuming and prone to errors, and ML-based approaches mainly focus on annotating unstructured free text sources rather than structured data like SIEM rules. Structured data often contains limited information, further complicating the annotation process and making it a challenging task. To address these challenges, we propose Rule-ATT&CK Mapper (RAM), a novel framework that leverages LLMs to automate the mapping of structured SIEM rules to MITRE ATT&CK techniques. RAM's multi-stage pipeline, which was inspired by the prompt chaining technique, enhances mapping accuracy without requiring LLM pre-training or fine-tuning. Using the Splunk Security Content dataset, we evaluate RAM's performance using several LLMs, including GPT-4-Turbo, Qwen, IBM Granite, and Mistral. Our evaluation highlights GPT-4-Turbo's superior performance, which derives from its enriched knowledge base, and an ablation study emphasizes the importance of external contextual knowledge in overcoming the limitations of LLMs' implicit knowledge for domain-specific tasks. These findings demonstrate RAM's potential in automating cybersecurity workflows and provide valuable insights for future advancements in this field.

Figures (4)

• Figure 1: Overview of our AI Agent-based RAM pipeline.
• Figure 2: An illustration of the different steps in RAM.
• Figure 3: Overview of prompt structure used in all steps of the pipeline.
• Figure 4: Distribution of length of labels in test samples.