Predicting Tail-Risk Escalation in IDS Alert Time Series

TL;DR

This work tackles the problem of forecasting tail-risk escalation in IDS alert streams by treating high-volume surge events as extreme regimes. It derives per-minute intensity, volatility, and momentum features from large Suricata IDS datasets and trains gradient-boosted trees (XGBoost) to predict, within a 30-minute horizon, whether the alert rate will cross the empirical $95^{th}$ percentile. The approach yields strong, stratum-aware predictive performance (accuracy and ROC–AUC consistently high) and reveals that raw intensity is the primary signal, with volatility and momentum providing meaningful gains, especially for high-severity alerts. A lightweight, interpretable visualization accompanies the model to convey early-warning risk across severities, supporting proactive SOC decision-making and resource planning, and the authors provide open-source models for replication and further research.

Abstract

Network defenders face a steady stream of attacks, observed as raw Intrusion Detection System (IDS) alerts. The sheer volume of alerts demands prioritization, typically based on high-level risk classifications. This work expands the scope of risk measurement by examining alerts not only through their technical characteristics but also by examining and classifying their temporal patterns. One critical issue in responding to intrusion alerts is determining whether an alert is part of an escalating attack pattern or an opportunistic scan. To identify the former, we apply extreme-regime forecasting methods from financial modeling to IDS data. Extreme-regime forecasting is designed to identify likely future high-impact events or significant shifts in system behavior. Using these methods, we examine attack patterns by computing per-minute alert intensity, volatility, and a short-term momentum measure derived from weighted moving averages. We evaluate the efficacy of a supervised learning model for forecasting future escalation patterns using these derived features. The trained model identifies future high-intensity attacks and demonstrates strong predictive performance, achieving approximately 91\% accuracy, 89\% recall, and 98\% precision. Our contributions provide a temporal measurement framework for identifying future high-intensity attacks and demonstrate the presence of predictive early-warning signals within the temporal structure of IDS alert streams. We describe our methods in sufficient detail to enable reproduction using other IDS datasets. In addition, we make the trained models openly available to support further research. Finally, we introduce an interpretable visualization that enables defenders to generate early predictive warnings of elevated volumetric arrival risk.

Figures (5)

• Figure 1: Temporal evolution of IDS alert intensity across severity strata. Each subplot shows the smoothed arrival intensity $\lambda_t$ for one severity class, with all subplots aligned on a common time axis for a 24-hour time slice. The night period shows a period of low activity. The plots also show heterogeneity in the temporal behavior of different severities.
• Figure 2: Regime overlays for the Major stratum. The top panel shows the alert intensity $\lambda(t)$ colored by inferred regimes (baseline intensity, buildup, surge), while the bottom scatter plot illustrates the corresponding regions in the volatility–momentum plane. Together, these visualizations highlight transitions from stable to escalated attack behavior.
• Figure 3: Proof-of-concept visualization of forecasted tail risk and cross-stratum risk comparison.Top panels: For each IDS alert stratum, the figure overlays the model-predicted probability of entering an extreme-intensity regime (blue) with the realized alert intensity (black, normalized for visualization). Shaded regions indicate time intervals labeled as extreme regimes under the 95th-percentile definition. Vertical dashed lines denote three selected time slices used for cross-sectional analysis. Bottom panels: Predicted tail-risk probabilities across all alert strata at the corresponding time slices, ordered from high-intensity, cooldown, to baseline operational states. Together, this proof-of-concept visualization demonstrates how temporal risk forecasts can simultaneously convey within-stratum escalation dynamics and relative risk distribution across strata at key operational moments.
• Figure 4: Volatility–momentum phase portrait of predicted tail risk. The figure visualizes the decision surface learned by the XGBoost model in the volatility–momentum feature space. Each point represents a one-minute observation, while the background shading indicates the predicted probability of transitioning into an extreme-intensity regime. The color bar reports the uncalibrated probability values produced by the classifier.
• Figure 5: Failure of HMMs in generalizing to clusters