Practitioners' Expectations on Log Anomaly Detection

TL;DR

This study investigates practitioners' expectations for log anomaly detection by combining 15 interviews, a broad online survey of $312$ practitioners across $36$ countries, and a literature review of $2014$–$2024$ premier-venue papers. It reveals widespread dissatisfaction with current log monitoring tools and a strong demand for automated detection that delivers high recall/precision, real-time performance, interpretability, generalizability across log structures, customization, and explicit privacy protections. The literature review shows that most proposed techniques rely on historical normal logs and overlook data like metrics and traces, event-level analysis, and explainability, contributing to a gap between research and practice. The authors call for research to prioritize interpretability, cross-domain generalization, user-friendly design, and privacy-aware customization to better meet practitioners' needs and encourage adoption of automated log anomaly detection tools.

Abstract

Log anomaly detection has become a common practice for software engineers to analyze software system behavior. Despite significant research efforts in log anomaly detection over the past decade, it remains unclear what are practitioners' expectations on log anomaly detection and whether current research meets their needs. To fill this gap, we conduct an empirical study, surveying 312 practitioners from 36 countries about their expectations on log anomaly detection. In particular, we investigate various factors influencing practitioners' willingness to adopt log anomaly detection tools. We then perform a literature review on log anomaly detection, focusing on publications in premier venues from 2014 to 2024, to compare practitioners' needs with the current state of research. Based on this comparison, we highlight the directions for researchers to focus on to develop log anomaly detection techniques that better meet practitioners' expectations.

Figures (10)

• Figure 1: The overview of the research methodology.
• Figure 2: The percentages of tool usage for log anomaly detection.
• Figure 3: Data resource availability for log anomaly detection.
• Figure 4: The current issues with log monitoring tools for log anomaly detection.
• Figure 5: The importance of automated log anomaly detection tools.