Attention and Autoencoder Hybrid Model for Unsupervised Online Anomaly Detection

TL;DR

This work tackles unsupervised online anomaly detection in time series by coupling an autoencoder (AE) with a transformer-based attention model. The AE encodes short windows to local latent representations, while a Transformer forecasts the next window’s latent embedding, enabling long-term temporal modeling without recurrent networks. Anomaly scoring combines reconstruction error patterns—either via a moving-average threshold or a moment-based criterion—applied to the predicted next window. Across six NAB datasets, the hybrid model delivers high precision, recall, and F1 scores, outperforms several baselines, and benefits from parallelizable architecture suitable for real-time deployment, as demonstrated by favorable training times on GPU hardware. The approach advances online anomaly detection by uniting local structure learning with long-range temporal forecasting in latent space.

Abstract

This paper introduces a hybrid attention and autoencoder (AE) model for unsupervised online anomaly detection in time series. The autoencoder captures local structural patterns in short embeddings, while the attention model learns long-term features, facilitating parallel computing with positional encoding. Unique in its approach, our proposed hybrid model combines attention and autoencoder for the first time in time series anomaly detection. It employs an attention-based mechanism, akin to the deep transformer model, with key architectural modifications for predicting the next time step window in the autoencoder's latent space. The model utilizes a threshold from the validation dataset for anomaly detection and introduces an alternative method based on analyzing the first statistical moment of error, improving accuracy without dependence on a validation dataset. Evaluation on diverse real-world benchmark datasets and comparing with other well-established models, confirms the effectiveness of our proposed model in anomaly detection.

Figures (4)

• Figure 1: Autoencoder architecture.
• Figure 2: The proposed transformer architecture.
• Figure 3: Schematic of the proposed model for anomaly detection.
• Figure 4: Experimental results for different time series.