LLM Assisted Anomaly Detection Service for Site Reliability Engineers: Enhancing Cloud Infrastructure Resilience

TL;DR

The paper presents a scalable, cloud-deployed Anomaly Detection Service with a generalizable API for industrial time-series data to assist Site Reliability Engineers in cloud infrastructure management. It combines a modular data-agnostic pipeline (including a DNN_AutoEncoder-based ReconstructAD) with Chi-Square derived $p$-values and PCA for multivariate scoring, and expands via LLM-assisted anomaly modelling to map components and failure modes to monitoring metrics. Benchmark results on public datasets (SMD, MSL, SMAP) show competitive F1 scores, while real-world usage is demonstrated by hundreds of thousands of API calls and wide adoption, underscoring practical impact. The work also outlines future directions toward time-series foundation models enabling zero-shot anomaly detection, promising broader resilience and faster incident response for cloud services.

Abstract

This paper introduces a scalable Anomaly Detection Service with a generalizable API tailored for industrial time-series data, designed to assist Site Reliability Engineers (SREs) in managing cloud infrastructure. The service enables efficient anomaly detection in complex data streams, supporting proactive identification and resolution of issues. Furthermore, it presents an innovative approach to anomaly modeling in cloud infrastructure by utilizing Large Language Models (LLMs) to understand key components, their failure modes, and behaviors. A suite of algorithms for detecting anomalies is offered in univariate and multivariate time series data, including regression-based, mixture-model-based, and semi-supervised approaches. We provide insights into the usage patterns of the service, with over 500 users and 200,000 API calls in a year. The service has been successfully applied in various industrial settings, including IoT-based AI applications. We have also evaluated our system on public anomaly benchmarks to show its effectiveness. By leveraging it, SREs can proactively identify potential issues before they escalate, reducing downtime and improving response times to incidents, ultimately enhancing the overall customer experience. We plan to extend the system to include time series foundation models, enabling zero-shot anomaly detection capabilities.

Figures (6)

• Figure 1: Overview of the end-to-end Cloud Metric Monitoring System using Anomaly Detection
• Figure 2: Mapping LLM-Generated Knowledge with the right dataset variables for building a more informed anomaly model for any industrial assets
• Figure 5: Anomaly Detection Service
• Figure 6: Continuous Visitor Journey (with KDD-2022 Anomaly Detection Tutorial with close to 200+ visits and 150+ participants
• Figure 7: Combined plot of API calls (line) and user statistics (bars).