THEMIS: Unlocking Pretrained Knowledge with Foundation Model Embeddings for Anomaly Detection in Time Series

TL;DR

THEMIS addresses the challenge of detecting anomalies in time series without labeled data by leveraging frozen embeddings from a forecasting foundation model (Chronos) and applying robust outlier detection on a Windowed Absolute Similarity Matrix. By decoupling representation learning from anomaly scoring and using modular adapters (Spectral Residual, LOF, Mean Similarity, Trimmed Top-k), it achieves state-of-the-art performance on the MSL benchmark and strong results on SMAP and SWaT in a zero-shot setting. The approach offers hyperparameter robustness and interpretability, highlighting the potential of pretrained representations for adaptable time series anomaly detection across domains. This framework paves the way for exploring additional foundation models and more sophisticated outlier detectors to improve real-world deployment.

Abstract

Time series anomaly detection forms a very crucial area in several domains but poses substantial challenges. Due to time series data possessing seasonality, trends, noise, and evolving patterns (concept drift), it becomes very difficult to set a general notion of what constitutes normal behavior. Anomalies themselves could be varied, ranging from a single outlier to contextual or collective anomalies, and are normally very rare; hence, the dataset is largely imbalanced. Additional layers of complexities arise due to the problems of increased dimensionality of modern time series, real-time detection criteria, setting up appropriate detection thresholds, and arriving at results that are interpretable. To embrace these multifaceted challenges, very strong, flexible, and interpretable approaches are required. This paper presents THEMIS, a new framework for time series anomaly detection that exploits pretrained knowledge from foundation models. THEMIS extracts embeddings from the encoder of the Chronos time series foundation model and applies outlier detection techniques like Local Outlier Factor and Spectral Decomposition on the self-similarity matrix, to spot anomalies in the data. Our experiments show that this modular method achieves SOTA results on the MSL dataset and performs quite competitively on the SMAP and SWAT$^*$ datasets. Notably, THEMIS exceeds models trained specifically for anomaly detection, presenting hyperparameter robustness and interpretability by default. This paper advocates for pretrained representations from foundation models for performing efficient and adaptable anomaly detection for time series data.

Figures (20)

• Figure 1: Architecture of the proposed anomaly detection system using TSFM embeddings and an outlier detection algorithm.
• Figure 2: Visualization of anomaly scores on a segment from the SMAP dataset. Left: Raw time series input with ground truth anomaly regions shaded in red. Right: Corresponding anomaly scores generated by Themis. The red horizontal line denotes the threshold. Anomalous regions are accurately identified with high anomaly scores, while non-anomalous regions exhibit low scores. Anomalous regions are shaded in red.
• Figure 3: Left:Error bar plots show the mean F1-score and standard deviation for different values of the spectral scoring hyperparameter $k$ (number of top eigenvectors used) on the MSL, SMAP, and SWaT$^*$ datasets. Themis consistently achieves stable and high F1-scores across a wide range of $k$ values. Right: Heatmap of the similarity matrix $\mathbf{S}$ for segment of SMAP shown in Figure \ref{['fig:msl_plot']}(Left). Anomalous points exhibit low similarity to normal regions.
• Figure 4: F1-scores (%) of spectral residual anomaly scoring on the MSL dataset across varying batch sizes$B \in \{1, 4, 16\}$and number of retained eigenvectors$k \in \{2, 5, 10, 15, 20\}$. Each curve corresponds to a fixed $k$. Larger batch sizes consistently yield improved detection performance, indicating the benefit of broader contextual similarity modeling.
• Figure 5: Input time series from the NAB dataset.