Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM-Assisted Logic Rule Learning: Scaling Human Expertise for Time Series Anomaly Detection

Haoting Zhang, Shekhar Jain

TL;DR

Addresses scalable, interpretable anomaly detection for Amazon's million-scale time series by bridging domain expertise with automated logic via LLMs. Proposes a three-stage pipeline: multimodal LLM-based labeling, LLM-driven rule generation with iterative refinement, and LLM-assisted semantic augmentation to classify anomalies. Demonstrates that the learned logic-based rules outperform unsupervised baselines and direct LLM detection, offering deterministic outputs with production-friendly latency, and shows robustness to distribution shifts through trajectory-aware learning. The framework provides a generalizable blueprint for scaling expert knowledge to large-scale operational tasks across domains.

Abstract

Time series anomaly detection is critical for supply chain management to take proactive operations, but faces challenges: classical unsupervised anomaly detection based on exploiting data patterns often yields results misaligned with business requirements and domain knowledge, while manual expert analysis cannot scale to millions of products in the supply chain. We propose a framework that leverages large language models (LLMs) to systematically encode human expertise into interpretable, logic-based rules for detecting anomaly patterns in supply chain time series data. Our approach operates in three stages: 1) LLM-based labeling of training data instructed by domain knowledge, 2) automated generation and iterative improvements of symbolic rules through LLM-driven optimization, and 3) rule augmentation with business-relevant anomaly categories supported by LLMs to enhance interpretability. The experiment results showcase that our approach outperforms the unsupervised learning methods in both detection accuracy and interpretability. Furthermore, compared to direct LLM deployment for time series anomaly detection, our approach provides consistent, deterministic results with low computational latency and cost, making it ideal for production deployment. The proposed framework thus demonstrates how LLMs can bridge the gap between scalable automation and expert-driven decision-making in operational settings.

LLM-Assisted Logic Rule Learning: Scaling Human Expertise for Time Series Anomaly Detection

TL;DR

Addresses scalable, interpretable anomaly detection for Amazon's million-scale time series by bridging domain expertise with automated logic via LLMs. Proposes a three-stage pipeline: multimodal LLM-based labeling, LLM-driven rule generation with iterative refinement, and LLM-assisted semantic augmentation to classify anomalies. Demonstrates that the learned logic-based rules outperform unsupervised baselines and direct LLM detection, offering deterministic outputs with production-friendly latency, and shows robustness to distribution shifts through trajectory-aware learning. The framework provides a generalizable blueprint for scaling expert knowledge to large-scale operational tasks across domains.

Abstract

Time series anomaly detection is critical for supply chain management to take proactive operations, but faces challenges: classical unsupervised anomaly detection based on exploiting data patterns often yields results misaligned with business requirements and domain knowledge, while manual expert analysis cannot scale to millions of products in the supply chain. We propose a framework that leverages large language models (LLMs) to systematically encode human expertise into interpretable, logic-based rules for detecting anomaly patterns in supply chain time series data. Our approach operates in three stages: 1) LLM-based labeling of training data instructed by domain knowledge, 2) automated generation and iterative improvements of symbolic rules through LLM-driven optimization, and 3) rule augmentation with business-relevant anomaly categories supported by LLMs to enhance interpretability. The experiment results showcase that our approach outperforms the unsupervised learning methods in both detection accuracy and interpretability. Furthermore, compared to direct LLM deployment for time series anomaly detection, our approach provides consistent, deterministic results with low computational latency and cost, making it ideal for production deployment. The proposed framework thus demonstrates how LLMs can bridge the gap between scalable automation and expert-driven decision-making in operational settings.
Paper Structure (10 sections, 3 figures, 3 tables, 1 algorithm)

This paper contains 10 sections, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology
  3. Labeling Stage
  4. Learning Stage
  5. Rule Prototype Generation
  6. Iterative Rule Improvement
  7. Complete Learning Framework for Logic-based Rules
  8. Augmentation Stage
  9. Experiment Results
  10. Conclusion

Figures (3)

  • Figure 1: A summarized three-stage framework of our approach: 1) instructing multimodal (vision-language) LLMs to label a training dataset, 2) LLM-driven automated learning of logic-based symbolic rules from the labeled dataset, and 3) employing LLMs to categorize detection rules to augment business interpretability. Our approach leverages LLMs to first encode human domain expertise into a labeled dataset and then distill it into logic-based rules.
  • Figure 2: A simplified illustration of iterative rule improvements: The performance evaluation based on the labeled dataset identifies behavioral patterns of the rule (over-conservative, over-aggressive, etc.) and LLMs apply targeted modifications to refine the rule, achieving the target performance.
  • Figure 3: The comparison between the learned rules and supervised learning models trained on the label dataset. Week 56 includes a holiday that causes demand spikes across products, degrading the performance of supervised learning models because of the high anomaly rate.