Automated Alert Classification and Triage (AACT): An Intelligent System for the Prioritisation of Cybersecurity Alerts

TL;DR

AACT addresses the problem of vast, noisy cybersecurity alerts overwhelming SOC analysts by learning in real time from human triage actions to automatically close benign alerts and prioritise suspicious ones. It combines dynamic, history-aware features with stable static alert attributes and trains a gradient-boosting classifier to predict analyst decisions, enabling near-real-time scoring and automatic closure when confidence is high. In real deployment, AACT achieved a 61% reduction in alerts over six months with a low false negative rate of $1.36\%$, while maintaining high predictive performance on both internal Sophos data and the public AIT Open Alert dataset. The approach is practical for managed and in-house SOCs, offering interpretable threat scores and scalable live scoring, with potential extensions to incident-centric prioritisation and tenant-aware weighting of analyst labels.

Abstract

Enterprise networks are growing ever larger with a rapidly expanding attack surface, increasing the volume of security alerts generated from security controls. Security Operations Centre (SOC) analysts triage these alerts to identify malicious activity, but they struggle with alert fatigue due to the overwhelming number of benign alerts. Organisations are turning to managed SOC providers, where the problem is amplified by context switching and limited visibility into business processes. A novel system, named AACT, is introduced that automates SOC workflows by learning from analysts' triage actions on cybersecurity alerts. It accurately predicts triage decisions in real time, allowing benign alerts to be closed automatically and critical ones prioritised. This reduces the SOC queue allowing analysts to focus on the most severe, relevant or ambiguous threats. The system has been trained and evaluated on both real SOC data and an open dataset, obtaining high performance in identifying malicious alerts from benign alerts. Additionally, the system has demonstrated high accuracy in a real SOC environment, reducing alerts shown to analysts by 61% over six months, with a low false negative rate of 1.36% over millions of alerts.

Figures (8)

• Figure 1: Daily alert counts across all tenants
• Figure 2: An overview of AACT
• Figure 3: Correlation matrix for the 'is investigated' dependent variable and the category investigation rates with lookback windows of $7$, $30$ and $60$ days.
• Figure 4: The precision versus recall curve at varying thresholds
• Figure 5: The ROC curve showing the false positive rate versus the true positive rate at varying thresholds