A Methodological Report on Anomaly Detection on Dynamic Knowledge Graphs

TL;DR

This approach significantly outperforms the baseline on the ISWC 2024 Dynamic Knowledge Graph Anomaly Detection dataset, providing a robust solution for anomaly detection in dynamic complex data.

Abstract

In this paper, we explore different approaches to anomaly detection on dynamic knowledge graphs, specifically in a Micro-services environment for Kubernetes applications. Our approach explores three dynamic knowledge graph representations: sequential data, hierarchical data and inter-service dependency data, with each representation incorporating increasingly complex structural information of dynamic knowledge graph. Different machine learning and deep learning models are tested on these representations. We empirically analyse their performance and propose an approach based on ensemble learning of these models. Our approach significantly outperforms the baseline on the ISWC 2024 Dynamic Knowledge Graph Anomaly Detection dataset, providing a robust solution for anomaly detection in dynamic complex data.

Figures (6)

• Figure 1: Left: The ontology overview of the relationships between Kubernetes concepts of the ADDKG dataset. Right: The simplified hierarchical structure of the ADDKG dataset. The container information is aggregated as the attributes of Pods. The Connections serve as attributes of the edges between Pods and Services.
• Figure 2: This figure presents the sequential data representation based on the simplified tree structure.
• Figure 3: This figure presents the hierarchical data representation based on the simplified tree structure.
• Figure 4: This figure presents the inter-service data representation based on the simplified tree structure.
• Figure 5: The ensemble learning framework used in this competition. The best-performing models: XGB, SVM, and SA are combined using three voting mechanisms: hard-unanimous, hard-majority, and soft voting, to generate the prediction $\hat{y_1}$, where $\hat{y_1} \in \{0, 1\}$, with 0 indicating normal and 1 indicating anomaly. Optionally, a two-stage model combination can be applied, where the unsupervised Isolation Forest further refines the anomalies detected in $\hat{y_1}$ to reduce false positives.