A unified framework for detecting point and collective anomalies in operating system logs via collaborative transformers

TL;DR

CoLog introduces a unified, multimodal framework for log anomaly detection by jointly modeling semantic and sequence modalities with collaborative transformers and a modality adaptation layer. It reframes log anomalies as negative sentiments and uses impressive cross-modal attention to capture interactions, enabling robust detection of both point and collective anomalies. The approach achieves state-of-the-art results across seven OS-log benchmarks, and extensive ablations validate the necessity of each component, including MHIA, MAL, and a latent fusion space. The work advances practical log analysis by delivering a reproducible, end-to-end solution ready for deployment in cybersecurity and system-monitoring contexts.

Abstract

Log anomaly detection is crucial for preserving the security of operating systems. Depending on the source of log data collection, various information is recorded in logs that can be considered log modalities. In light of this intuition, unimodal methods often struggle by ignoring the different modalities of log data. Meanwhile, multimodal methods fail to handle the interactions between these modalities. Applying multimodal sentiment analysis to log anomaly detection, we propose CoLog, a framework that collaboratively encodes logs utilizing various modalities. CoLog utilizes collaborative transformers and multi-head impressed attention to learn interactions among several modalities, ensuring comprehensive anomaly detection. To handle the heterogeneity caused by these interactions, CoLog incorporates a modality adaptation layer, which adapts the representations from different log modalities. This methodology enables CoLog to learn nuanced patterns and dependencies within the data, enhancing its anomaly detection capabilities. Extensive experiments demonstrate CoLog's superiority over existing state-of-the-art methods. Furthermore, in detecting both point and collective anomalies, CoLog achieves a mean precision of 99.63%, a mean recall of 99.59%, and a mean F1 score of 99.61% across seven benchmark datasets for log-based anomaly detection. The comprehensive detection capabilities of CoLog make it highly suitable for cybersecurity, system monitoring, and operational efficiency. CoLog represents a significant advancement in log anomaly detection, providing a sophisticated and effective solution to point and collective anomaly detection through a unified framework and a solution to the complex challenges automatic log data analysis poses. We also provide the implementation of CoLog at https://github.com/NasirzadehMoh/CoLog.

Figures (19)

• Figure 1: Illustration of how the sequence modality ($B_{1}$, $B_{2}$) adds more information to semantic modality ($e_{4}$, $e_{8}$) and how the same samples with different backgrounds are separated in a unified semantic space. For the $e_{4}$ and $e_{8}$ event vectors, their backgrounds, specifically $B_{1}$ and $B_{2}$, might be utilized to depict $e_{4}$ and $e_{8}$ in a more meaningful manner (it should be noted that $e_{4}$ is equivalent to $e_{8}$). In this sense, we designate the $B_{1}$ and $B_{2}$ as sequence modalities and the $e_{4}$ and $e_{8}$ as semantic modalities.
• Figure 2: A comprehensive overview of the existing literature dealing with the approaches for identifying anomalies in log data.
• Figure 3: (a) Several lines extracted from the BLueGene/L log dataset.; (b) Semantic modality: The semantic modality is constructed using the extracted semantic vectors from log events' messages.; (c) Sequence modality: The construction of the sequence modality involves appending semantic vectors into sequence vectors based on window sizes of 3, 4, 5, and 6.
• Figure 4: The overview of CoLog. Light green and gold colors demonstrate modality encoders. Each encoder in the collaborative transformer consists of MHIA, MLP, MAL, and LNs. MHIA and MAL are multi-head impressed attention and modality adaptation layer modules, respectively. The preprocess layer transforms unstructured logs into easily understandable data for the model. The purpose of the balancing layer is to regulate the influences of different modalities when calculating the final results.
• Figure 5: Illustration of differences between background and subsequent event vectors. Background and context sequence vectors are constructed based on background and subsequent event vectors. In mathematical terms, ${B}_{i}=[{V}_{{i}-{3}}^{sem},\ {V}_{{i}-{2}}^{sem},\ {V}_{{i}-{1}}^{sem}]$ and ${C}_{i}=[{V}_{{i}-{3}}^{sem},\ {V}_{{i}-{2}}^{sem},\ {V}_{{i}-{1}}^{sem},\ {V}_{{i}+{1}}^{sem},\ {V}_{{i}+{2}}^{sem},\ {V}_{{i}+{3}}^{sem}]$, where ${V}_{i}^{sem}$ is the semantic vector of the log message ${i}$ extracted by SBERT, ${B}_{i}$ is the background sequence vector of log message ${i}$, and ${C}_{i}$ is the context sequence vector of log message ${i}$.