Time Series Foundational Models: Their Role in Anomaly Detection and Prediction

TL;DR

This paper evaluates Time Series Foundational Models (TSFMs) for anomaly detection and prediction, highlighting concerns around generalizability, data leakage, interpretability, and computational cost. It conducts a systematic, dataset-wide comparison against traditional statistical and deep-learning baselines across five datasets, including series with no discernible patterns. The results show that while TSFMs can be extended to anomaly tasks, they rarely outperform specialized methods in accuracy or efficiency, and fine-tuning provides limited gains. The study underscores the need for task-focused designs, domain knowledge integration, multimodal data, and explainability techniques to make TSFMs practically useful for anomaly-focused applications.

Abstract

Time series foundational models (TSFM) have gained prominence in time series forecasting, promising state-of-the-art performance across various applications. However, their application in anomaly detection and prediction remains underexplored, with growing concerns regarding their black-box nature, lack of interpretability and applicability. This paper critically evaluates the efficacy of TSFM in anomaly detection and prediction tasks. We systematically analyze TSFM across multiple datasets, including those characterized by the absence of discernible patterns, trends and seasonality. Our analysis shows that while TSFMs can be extended for anomaly detection and prediction, traditional statistical and deep learning models often match or outperform TSFM in these tasks. Additionally, TSFMs require high computational resources but fail to capture sequential dependencies effectively or improve performance in few-shot or zero-shot scenarios. \noindent The preprocessed datasets, codes to reproduce the results and supplementary materials are available at https://github.com/smtmnfg/TSFM.

Figures (2)

• Figure 1: Overview of the analysis procedure. The analysis is categorized into three parts: (a) Foundational models pre-trained from scratch specifically for time series analysis, including TimeGPT garza2023timegpt, Time-MOE shi2024time, and MOIRAI gerald2024unified, all of which can be fine-tuned for additional datasets; (b) Models that adapt large language models (LLMs) for time series tasks, namely FPT zhou2023one and Chronos ansari2024chronos, where the LLM components remain frozen, with Chronos supporting fine-tuning for other datasets, unlike FPT; and (c) Baseline models trained from scratch for anomaly detection and prediction tasks. The computational cost of all foundational, statistical, and deep learning models used is also evaluated.
• Figure 2: Selected EDA plots for Pulp and Paper Manufacturing Dataset and FF Dataset.