Towards Resource-Efficient Federated Learning in Industrial IoT for Multivariate Time Series Analysis

TL;DR

A novel compression-based optimization problem is proposed at the server-side of a FL paradigm that fusses the received local models broadcast and performs pruning generating a more compressed model, which is expected to have significant benefits towards reducing the processing, storage and communication complexity.

Abstract

Anomaly and missing data constitute a thorny problem in industrial applications. In recent years, deep learning enabled anomaly detection has emerged as a critical direction, however the improved detection accuracy is achieved with the utilization of large neural networks, increasing their storage and computational cost. Moreover, the data collected in edge devices contain user privacy, introducing challenges that can be successfully addressed by the privacy-preserving distributed paradigm, known as federated learning (FL). This framework allows edge devices to train and exchange models increasing also the communication cost. Thus, to deal with the increased communication, processing and storage challenges of the FL based deep anomaly detection NN pruning is expected to have significant benefits towards reducing the processing, storage and communication complexity. With this focus, a novel compression-based optimization problem is proposed at the server-side of a FL paradigm that fusses the received local models broadcast and performs pruning generating a more compressed model. Experiments in the context of anomaly detection and missing value imputation demonstrate that the proposed FL scenario along with the proposed compressed-based method are able to achieve high compression rates (more than $99.7\%$) with negligible performance losses (less than $1.18\%$ ) as compared to the centralized solutions.

Figures (4)

• Figure 1: The proposed resource-efficient federated learning protocol, where each edge device contains sensors that measure different physical quantities, thus having different feature space. In this illustration, each edge device has access to only one sensor ($\boldsymbol{M_i} =\boldsymbol{1}$).
• Figure 2: The autoencoder-based model employed by the edge devices utilizing the pre-processed local datasets derived from the time window methodology.
• Figure 3: The Federated Learning classical scenario (FL-multivariate), where each edge device contains the same type of sensors that measure the exact same quantities.
• Figure 4: Model complexities comparison of our proposed federated learning scenario (FL-univariate) with and without the compression scheme, the classical FL scenario with and without the compression scheme and the centralized solution.