Federated Learning with Reservoir State Analysis for Time Series Anomaly Detection

TL;DR

This work tackles privacy and computational constraints in time-series anomaly detection by integrating Mahalanobis distance on reservoir states (MD-RS) with Incremental Federated Learning (IncFed MD-RS). The approach models normal reservoir states as a Gaussian distribution and detects anomalies via the Mahalanobis distance, while aggregating client-side statistics to produce a global, equivalent-to-centralized model. Key contributions include the first federated reservoir-computing method for anomaly detection, theoretical and empirical equivalence to centralized MD-RS, robustness to short and heterogeneous client data, and a subsampling strategy to reduce communication and computation. The method achieves superior performance over deep-learning and reservoir-based baselines across multiple datasets, with practical gains in scalability and privacy, and code is publicly available.

Abstract

With a growing data privacy concern, federated learning has emerged as a promising framework to train machine learning models without sharing locally distributed data. In federated learning, local model training by multiple clients and model integration by a server are repeated only through model parameter sharing. Most existing federated learning methods assume training deep learning models, which are often computationally demanding. To deal with this issue, we propose federated learning methods with reservoir state analysis to seek computational efficiency and data privacy protection simultaneously. Specifically, our method relies on Mahalanobis Distance of Reservoir States (MD-RS) method targeting time series anomaly detection, which learns a distribution of reservoir states for normal inputs and detects anomalies based on a deviation from the learned distribution. Iterative updating of statistical parameters in the MD-RS enables incremental federated learning (IncFed MD-RS). We evaluate the performance of IncFed MD-RS using benchmark datasets for time series anomaly detection. The results show that IncFed MD-RS outperforms other federated learning methods with deep learning and reservoir computing models particularly when clients' data are relatively short and heterogeneous. We demonstrate that IncFed MD-RS is robust against reduced sample data compared to other methods. We also show that the computational cost of IncFed MD-RS can be reduced by subsampling from the reservoir states without performance degradation. The proposed method is beneficial especially in anomaly detection applications where computational efficiency, algorithm simplicity, and low communication cost are required.

Figures (6)

• Figure 1: Schematic illustrations of federated learning schemes. (a) Federated Averaging. Each client $c$ sends its local update $\mathbf{w_c}$ to the server. Then the server aggregates all local updates to construct a global model and sends it back to the clients. This procedure continues until training losses converge. (b) Incremental Federated Learning with ESN. Each client $c$ sends its matrices containing information on reservoir states and target values. Then the server calculates output weight matrix $\mathbf{W}_\mathrm{out}$ and sends it back to the clients. (c) Incremental Federated Learning with MD-RS. Each client $c$ sends its covariance matrix of reservoir state vectors. Then the server calculates the precision matrix by inverse calculation of the aggregated covariance matrix and sends it back to the clients.
• Figure 2: Overview of Mahalanobis Distance of Reservoir States.
• Figure 3: Distributions of the evaluation metric scores over time series in the test dataset in Server Machine Dataset (SMD). The triangle marks present the average evaluation metric scores.
• Figure 4: Distributions of the evaluation metric scores over time series in the test dataset in Soil Moisture Active Passive satellite (SMAP). The triangle marks present the average evaluation metric scores.
• Figure 5: Various performance comparisons on Pooled Server Metrics (PSM). (a) Performance of each method for different training data proportions. (b) Performance of each method with the varying number of clients.