CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection

TL;DR

A novel end-to-end self-supervised ContrAstive Representation Learning approach for time series Anomaly detection (CARLA), which leverages existing generic knowledge about time series anomalies and injects various types of anomalies as negative samples.

Abstract

One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. The normal boundary is often defined tightly, resulting in slight deviations being classified as anomalies, consequently leading to a high false positive rate and a limited ability to generalise normal patterns. To address this, we introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series Anomaly detection (CARLA). While existing contrastive learning methods assume that augmented time series windows are positive samples and temporally distant windows are negative samples, we argue that these assumptions are limited as augmentation of time series can transform them to negative samples, and a temporally distant window can represent a positive sample. Our contrastive approach leverages existing generic knowledge about time series anomalies and injects various types of anomalies as negative samples. Therefore, CARLA not only learns normal behaviour but also learns deviations indicating anomalies. It creates similar representations for temporally closed windows and distinct ones for anomalies. Additionally, it leverages the information about representations' neighbours through a self-supervised approach to classify windows based on their nearest/furthest neighbours to further enhance the performance of anomaly detection. In extensive tests on seven major real-world time series anomaly detection datasets, CARLA shows superior performance over state-of-the-art self-supervised and unsupervised TSAD methods. Our research shows the potential of contrastive representation learning to advance time series anomaly detection.

Figures (9)

• Figure 1: Histograms of the distribution of anomaly scores produced by (\ref{['fig:thoc-dist']}) THOC shen2020timeseries, (\ref{['fig:ts2vec-dist']}) and TS2Vec yue2022ts2vec and (\ref{['fig:carla-dist']}) CARLA models using M-6 dataset of the MSL benchmark hundman2018detecting.
• Figure 2: The end-to-end pipeline of CARLA consists of two main stages: the Pretext Stage and the Self-supervised Classification Stage. In the Pretext Stage, anomaly injection techniques are used for self-supervised learning. The Self-supervised Classification Stage integrates the learned representations for a contrastive approach that maximises the similarity between anchor samples and their nearest neighbours while minimising the similarity between anchor samples and their furthest neighbours. The output is a trained model and the majority class, enabling inference for anomaly detection.
• Figure 3: Different types of synthetic anomaly injection used in CARLA. The figure presents the effect of synthetic anomaly injections into a randomly selected window of size 400 from the first dimension of entity E-2 in the MSL dataset hundman2018detecting. The (\ref{['fig:original_window']}) represents the original time series window, while the remaining five demonstrate the same window but with different types of anomalies injected. The anomalies are categorised into (\ref{['fig:anomlay_Global']}) Global, (\ref{['fig:anomlay_Contextual']}) Contextual, (\ref{['fig:anomlay_Seasonal']}) Seasonal, (\ref{['fig:anomlay_Shapelet']}) Shapelet, and (\ref{['fig:anomlay_Trend']}) Trend. In each of the five plots with anomalies, the anomalous points or subsequences are accentuated with a red colour.
• Figure 4: Critical difference diagrams for multivariate (\ref{['fig:cdd-m']}) and univariate (\ref{['fig:cdd-u']}) time series benchmark datasets.
• Figure 5: Models' performance comparison on MTS & UTS datasets regarding their FPR and AU-PR. Models in the optimal quadrant (low FPR, high AU-PR) have superior anomaly detection with minimal false alarms.