Modeling Behavioral Preferences of Cyber Adversaries Using Inverse Reinforcement Learning

TL;DR

The paper tackles the challenge of inferring adversary intent when attacker tools and techniques continually evolve by modeling attacker behavior as an agent in a host-level Markov decision process and applying inverse reinforcement learning to reconstruct latent preferences from provenance-graph derived trajectories. It couples system-level audit logs with ATT&CK-based template graphs to map low-level events to high-level actions, enabling a linear reward framework R(s,a)=\sum_i w_i \phi_i(s,a) whose weights are learned via MAP-BIRL and MLE-IRL. The main contributions include an end-to-end pipeline from provenance graphs to IRL-based preference learning, a formal host-threat MDP, and empirical validation on CADETS and THEIA datasets showing consistent, often invariant attacker preferences that serve as behavioral signatures for attribution and defense planning. The findings demonstrate that low-level forensics data can reveal attacker predispositions, offering a scalable, tool-agnostic dimension to threat modeling with practical implications for threat attribution and defense design.

Abstract

This paper presents a holistic approach to attacker preference modeling from system-level audit logs using inverse reinforcement learning (IRL). Adversary modeling is an important capability in cybersecurity that lets defenders characterize behaviors of potential attackers, which enables attribution to known cyber adversary groups. Existing approaches rely on documenting an ever-evolving set of attacker tools and techniques to track known threat actors. Although attacks evolve constantly, attacker behavioral preferences are intrinsic and less volatile. Our approach learns the behavioral preferences of cyber adversaries from forensics data on their tools and techniques. We model the attacker as an expert decision-making agent with unknown behavioral preferences situated in a computer host. We leverage attack provenance graphs of audit logs to derive a state-action trajectory of the attack. We test our approach on open datasets of audit logs containing real attack data. Our results demonstrate for the first time that low-level forensics data can automatically reveal an adversary's subjective preferences, which serves as an additional dimension to modeling and documenting cyber adversaries. Attackers' preferences tend to be invariant despite their different tools and indicate predispositions that are inherent to the attacker. As such, these inferred preferences can potentially serve as unique behavioral signatures of attackers and improve threat attribution.

Figures (11)

• Figure 1: Provenance graphs aid forensic investigations by causally connecting related entities even when they are temporally distant in logs.
• Figure 2: The template graphs represent tactics and techniques in the ATT&CK matrix. We utilize these templates to identify attacker actions in the provenance graph using subgraph isomorphism.
• Figure 3: The reward functions inferred from the trajectories of different attackers using MAP-BIRL show their behavioral preferences.
• Figure 4: The state-action trajectory for the CADETS-2 attack
• Figure 5: The state-action trajectory for the CADETS-1 attack

Theorems & Definitions (1)

• Definition 1: Attack scenario