AD-NEv++ : The multi-architecture neuroevolution-based multivariate anomaly detection framework

TL;DR

This work tackles multivariate time-series anomaly detection under label scarcity and proposes AD-NEv++, a multi-architecture neuroevolution framework that optimizes both feature subspaces and neural architectures. It extends the prior AD-NEv by adding graph autoencoders and optional layers such as attention, skip, and dense connections, organized into a three-stage process: subspace evolution, model evolution, and non-gradient fine-tuning. The approach uses a two-population evolution for models and subspaces and a non-gradient weight mutation strategy for fine-tuning, with a reconstruction-loss objective for training. Empirical results on benchmark datasets (WADI, SWAT, MSL, SMAP, SMD) show AD-NEv++ consistently outperforms state-of-the-art deep learning and neuroevolution baselines, including improved GNN performance.

Abstract

Anomaly detection tools and methods enable key analytical capabilities in modern cyberphysical and sensor-based systems. Despite the fast-paced development in deep learning architectures for anomaly detection, model optimization for a given dataset is a cumbersome and time-consuming process. Neuroevolution could be an effective and efficient solution to this problem, as a fully automated search method for learning optimal neural networks, supporting both gradient and non-gradient fine tuning. However, existing frameworks incorporating neuroevolution lack of support for new layers and architectures and are typically limited to convolutional and LSTM layers. In this paper we propose AD-NEv++, a three-stage neuroevolution-based method that synergically combines subspace evolution, model evolution, and fine-tuning. Our method overcomes the limitations of existing approaches by optimizing the mutation operator in the neuroevolution process, while supporting a wide spectrum of neural layers, including attention, dense, and graph convolutional layers. Our extensive experimental evaluation was conducted with widely adopted multivariate anomaly detection benchmark datasets, and showed that the models generated by AD-NEv++ outperform well-known deep learning architectures and neuroevolution-based approaches for anomaly detection. Moreover, results show that AD-NEv++ can improve and outperform the state-of-the-art GNN (Graph Neural Networks) model architecture in all anomaly detection benchmarks.

Figures (2)

• Figure 1: AD-NEv++ framework architecture. Time series data is used to train and evaluate a set of models during neuroevolution through different levels. The framework returns an optimized ensemble model based on an initially specified model architecture supporting choices at different levels of abstraction: basic model (A0), specific layers (A1), and optional layers (A2). The framework extends and generalizes AD-NEv's optimization workflow with support for a larger class of models, including graph auto-encoders and layers such as Attention, Skip-Connection, and Dense-Connection.
• Figure 2: Addition of skip connections to a vanilla auto-encoder model with convolutional (CNN) layers.