See it, Think it, Sorted: Large Multimodal Models are Few-shot Time Series Anomaly Analyzers

TL;DR

This work introduces a pioneering framework called the Time Series Anomaly Multimodal Analyzer (TAMA), which leverages the power of Large Multimodal Models (LMMs) to enhance both the detection and interpretation of anomalies in time series data.

Abstract

Time series anomaly detection (TSAD) is becoming increasingly vital due to the rapid growth of time series data across various sectors. Anomalies in web service data, for example, can signal critical incidents such as system failures or server malfunctions, necessitating timely detection and response. However, most existing TSAD methodologies rely heavily on manual feature engineering or require extensive labeled training data, while also offering limited interpretability. To address these challenges, we introduce a pioneering framework called the Time Series Anomaly Multimodal Analyzer (TAMA), which leverages the power of Large Multimodal Models (LMMs) to enhance both the detection and interpretation of anomalies in time series data. By converting time series into visual formats that LMMs can efficiently process, TAMA leverages few-shot in-context learning capabilities to reduce dependence on extensive labeled datasets. Our methodology is validated through rigorous experimentation on multiple real-world datasets, where TAMA consistently outperforms state-of-the-art methods in TSAD tasks. Additionally, TAMA provides rich, natural language-based semantic analysis, offering deeper insights into the nature of detected anomalies. Furthermore, we contribute one of the first open-source datasets that includes anomaly detection labels, anomaly type labels, and contextual description, facilitating broader exploration and advancement within this critical field. Ultimately, TAMA not only excels in anomaly detection but also provides a comprehensive approach for understanding the underlying causes of anomalies, pushing TSAD forward through innovative methodologies and insights.

Figures (7)

• Figure 1: Comparison of AUC-PR and F1 with PA on the UCR dataset. Models include the machine learning (IF, LOF), the deep learning (TranAD, MAD_GAN, MTAD_GAT), and LMMs (GPT-4o, Gemini-1.5-pro)
• Figure 2: Our framework converts time series into images for visual interpretation (“See it”). Then, LMMs are employed to analyze the visualized time series through Multimodal Reference Learning, Multimodal Analyzing, and Multi-scaled Self-reflection, ensuring self-consistency and stability in the analysis ("Think it"). Finally, the detected anomalous intervals are processed into the output format required for TSAD, providing descriptions and possible reasons for each anomaly ("Sorted").
• Figure 3: The AUC-PR of all models at various point-adjustment threshold $\alpha$ (PAT, defined in Appendix \ref{['sec:Appendix-PA-metrics']}).
• Figure 4: Results (without PA) of reference number ablation experiments.
• Figure 5: Results of window size ablation experiments. For the period of two datasets, $T_{UCR} \approx 200$, $T_{NASA-SMAP} \approx 100$