Try with Simpler -- An Evaluation of Improved Principal Component Analysis in Log-based Anomaly Detection

TL;DR

The paper investigates whether traditional log-based anomaly detection can approach state-of-the-art DL performance through simple adaptations. It introduces SemPCA, which injects a lightweight TF-IDF semantic representation into PCA to mitigate unseen log events, enabling effective and efficient anomaly detection across five datasets. Empirical results show SemPCA achieving comparable F1 scores to supervised/semi-supervised DL methods while delivering substantially better stability and training/prediction efficiency. The work supports the practical value of optimizing traditional techniques and provides a reusable toolbox for evaluating log-based anomaly detection methods.

Abstract

The rapid growth of deep learning (DL) has spurred interest in enhancing log-based anomaly detection. This approach aims to extract meaning from log events (log message templates) and develop advanced DL models for anomaly detection. However, these DL methods face challenges like heavy reliance on training data, labels, and computational resources due to model complexity. In contrast, traditional machine learning and data mining techniques are less data-dependent and more efficient but less effective than DL. To make log-based anomaly detection more practical, the goal is to enhance traditional techniques to match DL's effectiveness. Previous research in a different domain (linking questions on Stack Overflow) suggests that optimized traditional techniques can rival state-of-the-art DL methods. Drawing inspiration from this concept, we conducted an empirical study. We optimized the unsupervised PCA (Principal Component Analysis), a traditional technique, by incorporating lightweight semantic-based log representation. This addresses the issue of unseen log events in training data, enhancing log representation. Our study compared seven log-based anomaly detection methods, including four DL-based, two traditional, and the optimized PCA technique, using public and industrial datasets. Results indicate that the optimized unsupervised PCA technique achieves similar effectiveness to advanced supervised/semi-supervised DL methods while being more stable with limited training data and resource-efficient. This demonstrates the adaptability and strength of traditional techniques through small yet impactful adaptations.

Figures (5)

• Figure 1: An illustrating example for log terminology
• Figure 2: F1-score under different sets of training data
• Figure 3: Workflow of SemPCA
• Figure 4: An anomalous log sequence extracted from BGL. Line 9 of log sequence (a) is labeled by experts as anomalous log message.
• Figure 5: An anomalous log sequence extracted from BGL. The line 13 is labeled by experts as anomalous log message.