PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection

TL;DR

This paper tackles decentralized time series anomaly detection by introducing PeFAD, a PLM-based, parameter-efficient federated framework. It combines patch-based input with an anomaly-driven mask selection strategy, a privacy-preserving shared dataset synthesis mechanism, and knowledge distillation to address data scarcity and client heterogeneity. The approach freezes most PLM parameters and only tunes a small subset to reduce communication while maintaining performance, achieving state-of-the-art results on four real datasets and showing robustness in ablation and tuning studies. The work’s practical significance lies in enabling privacy-preserving, efficient, and accurate anomaly detection across distributed edge devices in diverse domains.

Abstract

With the proliferation of mobile sensing techniques, huge amounts of time series data are generated and accumulated in various domains, fueling plenty of real-world applications. In this setting, time series anomaly detection is practically important. It endeavors to identify deviant samples from the normal sample distribution in time series. Existing approaches generally assume that all the time series is available at a central location. However, we are witnessing the decentralized collection of time series due to the deployment of various edge devices. To bridge the gap between the decentralized time series data and the centralized anomaly detection algorithms, we propose a Parameter-efficient Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. PeFAD for the first time employs the pre-trained language model (PLM) as the body of the client's local model, which can benefit from its cross-modality knowledge transfer capability. To reduce the communication overhead and local model adaptation cost, we propose a parameter-efficient federated training module such that clients only need to fine-tune small-scale parameters and transmit them to the server for update. PeFAD utilizes a novel anomaly-driven mask selection strategy to mitigate the impact of neglected anomalies during training. A knowledge distillation operation on a synthetic privacy-preserving dataset that is shared by all the clients is also proposed to address the data heterogeneity issue across clients. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.

Figures (11)

• Figure 2: PeFAD framework overview. PeFAD consists of PLM-based local training and parameter-efficient federated training.
• Figure 3: Ablation study results of PeFAD and its variants
• Figure 4: Effect of various PLMs on model performance
• Figure 5: Parameter sensitivity analysis on SMD dataset
• Figure 6: The example of data synthesis, time series reconstruction and anomaly detection within the client from SMD dataset.

Theorems & Definitions (1)

• definition 1: Time Series