Graph Mixture of Experts and Memory-augmented Routers for Multivariate Time Series Anomaly Detection

TL;DR

The paper tackles unsupervised anomaly detection in multivariate time series by introducing Graph-MoE, a graph neural network framework that assigns a dedicated expert to each GNN layer to capture hierarchical inter-entity relationships. It further adds Memory-augmented Routers to adaptively weigh and integrate multi-layer representations using global historical temporal features, enabling effective inter-level aggregation. The approach is plug-and-play and demonstrates state-of-the-art AUROC performance on five public benchmarks (SWaT, WADI, PSM, MSL, SMD), with ablations highlighting the contribution of both the MoE and memory components. By modeling both short-range and long-range dependencies across layers and time, the method advances unsupervised anomaly detection in complex multivariate time series with practical impact for industrial and real-world systems.

Abstract

Multivariate time series (MTS) anomaly detection is a critical task that involves identifying abnormal patterns or events in data that consist of multiple interrelated time series. In order to better model the complex interdependence between entities and the various inherent characteristics of each entity, the GNN based methods are widely adopted by existing methods. In each layer of GNN, node features aggregate information from their neighboring nodes to update their information. In doing so, from shallow layer to deep layer in GNN, original individual node features continue to be weakened and more structural information,i.e., from short-distance neighborhood to long-distance neighborhood, continues to be enhanced. However, research to date has largely ignored the understanding of how hierarchical graph information is represented and their characteristics that can benefit anomaly detection. Existing methods simply leverage the output from the last layer of GNN for anomaly estimation while neglecting the essential information contained in the intermediate GNN layers. To address such limitations, in this paper, we propose a Graph Mixture of Experts (Graph-MoE) network for multivariate time series anomaly detection, which incorporates the mixture of experts (MoE) module to adaptively represent and integrate hierarchical multi-layer graph information into entity representations. It is worth noting that our Graph-MoE can be integrated into any GNN-based MTS anomaly detection method in a plug-and-play manner. In addition, the memory-augmented routers are proposed in this paper to capture the correlation temporal information in terms of the global historical features of MTS to adaptively weigh the obtained entity representations to achieve successful anomaly estimation. Extensive experiments on five challenging datasets prove the superiority of our approach and each proposed module.

Figures (5)

• Figure 1: Graph structure of two real-data in MTGFlow zhou2023detecting. Each node indicates an entity of multivariate time-series signals of a certain time step.
• Figure 2: The overview of our proposed Graph-MoE network. It mainly consists of 1) graph mixture of experts and 2) memory-augmented routers.
• Figure 3: The framework of our memory-augmented routers.
• Figure 4: Distribution of anomaly scores for normal and anomaly data across GANF, MTGFlow, and Graph-MoE models.
• Figure 5: Anomaly detection over time by our Graph-MoE.