ADALog: Adaptive Unsupervised Anomaly detection in Logs with Self-attention Masked Language Model

TL;DR

ADALog introduces an adaptive, unsupervised anomaly detection framework that operates directly on raw, unparsed logs using a masked language modeling objective with a DistilBERT backbone. By training solely on normal data and applying a percentile-based threshold derived from normal scores, it avoids rigid heuristics and parsing requirements while adapting to evolving log patterns. Empirical results on BGL, Thunderbird, and Spirit demonstrate strong generalization and competitive performance, with ablations highlighting the importance of modest masking, domain-tuned fine-tuning, and intra-log token-position cues for interpretability. The work offers a practical, scalable approach for real-world log anomaly detection and suggests future hybrid cloud-edge deployments to enable real-time applicability.

Abstract

Modern software systems generate extensive heterogeneous log data with dynamic formats, fragmented event sequences, and varying temporal patterns, making anomaly detection both crucial and challenging. To address these complexities, we propose ADALog, an adaptive, unsupervised anomaly detection framework designed for practical applicability across diverse real-world environments. Unlike traditional methods reliant on log parsing, strict sequence dependencies, or labeled data, ADALog operates on individual unstructured logs, extracts intra-log contextual relationships, and performs adaptive thresholding on normal data. The proposed approach utilizes a transformer-based, pretrained bidirectional encoder with a masked language modeling task, fine-tuned on normal logs to capture domain-specific syntactic and semantic patterns essential for accurate anomaly detection. Anomalies are identified via token-level reconstruction probabilities, aggregated into log-level scores, with adaptive percentile-based thresholding calibrated only on normal data. This allows the model to dynamically adapt to evolving system behaviors while avoiding rigid, heuristic-based thresholds common in traditional systems. We evaluate ADALog on benchmark datasets BGL, Thunderbird, and Spirit, showing strong generalization and competitive performance compared to state-of-the-art supervised and unsupervised methods. Additional ablation studies examine the effects of masking, fine-tuning, and token positioning on model behavior and interpretability.

Figures (3)

• Figure 1: Overview of proposed ADALog adaptive unsupervised anomaly detection in logs framework.
• Figure 2: Impact of different token masking strategies on anomaly detection performance across three benchmark log datasets.
• Figure 3: Heatmap of token probabilities for BGL, Thunderbird, and Spirit logs