MA-VAE: Multi-head Attention-based Variational Autoencoder Approach for Anomaly Detection in Multivariate Time-series Applied to Automotive Endurance Powertrain Testing

TL;DR

A variational autoencoder with multi-head attention (MA-VAE), which, when trained on unlabelled data, not only provides very few false positives but also manages to detect the majority of the anomalies presented, which offers a novel way to avoid the bypass phenomenon.

Abstract

A clear need for automatic anomaly detection applied to automotive testing has emerged as more and more attention is paid to the data recorded and manual evaluation by humans reaches its capacity. Such real-world data is massive, diverse, multivariate and temporal in nature, therefore requiring modelling of the testee behaviour. We propose a variational autoencoder with multi-head attention (MA-VAE), which, when trained on unlabelled data, not only provides very few false positives but also manages to detect the majority of the anomalies presented. In addition to that, the approach offers a novel way to avoid the bypass phenomenon, an undesirable behaviour investigated in literature. Lastly, the approach also introduces a new method to remap individual windows to a continuous time series. The results are presented in the context of a real-world industrial data set and several experiments are undertaken to further investigate certain aspects of the proposed model. When configured properly, it is 9% of the time wrong when an anomaly is flagged and discovers 67% of the anomalies present. Also, MA-VAE has the potential to perform well with only a fraction of the training and validation subset, however, to extract it, a more sophisticated threshold estimation method is required.

Figures (8)

• Figure 1: Features of a normal (black) and an anomalous (red) cycle plotted with respect to time. The anomalous cycle plotted represents a scenario where the wheel diameter has not been set correctly. The amplitude axis is z-score normalised to comply with confidentiality guidelines.
• Figure 2: An illustration of the proposed MA-VAE model. Blue shapes designate trainable models, orange deterministic tensors and green distribution parameters. The shape of each tensor is designated below it. During training $\textbf{Z}$ is used as the value matrix, denoted by the solid arrow, whereas during inference $\boldsymbol{\mu}_\textbf{Z}$ is used as the value matrix, denoted by the traced arrow.
• Figure 3: Precision-recall curves for the model variation without MA and MA-VAE.
• Figure 4: Precision-recall curves for the model trained on different training/validation subset sizes.
• Figure 5: Precision-recall curves for the model trained on different seeds.