Nested Dirichlet models for unsupervised attack pattern detection in honeypot data

TL;DR

The paper addresses the problem of unsupervised attack-pattern detection in honeypot data by developing nested Dirichlet models that cluster sessions by latent intents. It introduces constrained CBC, nested NCBC, and parent-child PCNBC frameworks, with extensions to unbounded topic and vocabulary sizes via GEM/Dirichlet processes. Key contributions include demonstrating that CBC can uncover rare MIRAI variants (e.g., MinerFinder), NCBC provides superior fit and interpretability through command-level topics, and GEM-based nonparametric priors enable adaptation to evolving attack vectors. The methods are validated on Imperial College London honeypot data, offering a practical, interpretable toolkit for threat-hunting and anomaly detection in cyber-security contexts.

Abstract

Cyber-systems are under near-constant threat from intrusion attempts. Attacks types vary, but each attempt typically has a specific underlying intent, and the perpetrators are typically groups of individuals with similar objectives. Clustering attacks appearing to share a common intent is very valuable to threat-hunting experts. This article explores Dirichlet distribution topic models for clustering terminal session commands collected from honeypots, which are special network hosts designed to entice malicious attackers. The main practical implications of clustering the sessions are two-fold: finding similar groups of attacks, and identifying outliers. A range of statistical models are considered, adapted to the structures of command-line syntax. In particular, concepts of primary and secondary topics, and then session-level and command-level topics, are introduced into the models to improve interpretability. The proposed methods are further extended in a Bayesian nonparametric fashion to allow unboundedness in the vocabulary size and the number of latent intents. The methods are shown to discover an unusual MIRAI variant which attempts to take over existing cryptocurrency coin-mining infrastructure, not detected by traditional topic-modelling approaches.

Figures (17)

• Figure 1: Cartoon representation of the full Nested Constrained Bayesian Clustering (NCBC) model.
• Figure 2: Cartoon representation of the Parent-Child Nested Bayesian Clustering (PCNBC) model.
• Figure 3: Cumulative number of unique words in the vocabulary $\mathcal{V}$ in the Imperial College London honeypot data, before and after preprocessing. The preprocessing steps are described in Section \ref{['sec:preprocessing']}.
• Figure 4: Estimated topic frequencies and estimated distribution of the number of non-empty topics $K_\varnothing$ under the CBC model in \ref{['mod1']}, fitted on the ICL honeypot data.
• Figure 5: Heatmaps of Jensen-Shannon divergences between the session-level word distributions, Jaccard coefficients for the estimated session-level groups, and difference in entropy of the estimated topic distributions under CBC without secondary topic and CBC with secondary topic, after alignment via the Hungarian algorithm.