Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhanced Federated Anomaly Detection Through Autoencoders Using Summary Statistics-Based Thresholding

Sofiane Laridi, Gregory Palmer, Kam-Ming Mark Tam

TL;DR

This study proposes a federated threshold calculation method that aggregates summary statistics from normal and anomalous data across clients to create a global threshold for Anomaly Detection with federated Autoencoders, enhancing detection accuracy and robustness while ensuring privacy.

Abstract

In Federated Learning (FL), anomaly detection (AD) is a challenging task due to the decentralized nature of data and the presence of non-IID data distributions. This study introduces a novel federated threshold calculation method that leverages summary statistics from both normal and anomalous data to improve the accuracy and robustness of anomaly detection using autoencoders (AE) in a federated setting. Our approach aggregates local summary statistics across clients to compute a global threshold that optimally separates anomalies from normal data while ensuring privacy preservation. We conducted extensive experiments using publicly available datasets, including Credit Card Fraud Detection, Shuttle, and Covertype, under various data distribution scenarios. The results demonstrate that our method consistently outperforms existing federated and local threshold calculation techniques, particularly in handling non-IID data distributions. This study also explores the impact of different data distribution scenarios and the number of clients on the performance of federated anomaly detection. Our findings highlight the potential of using summary statistics for threshold calculation in improving the scalability and accuracy of federated anomaly detection systems.

Enhanced Federated Anomaly Detection Through Autoencoders Using Summary Statistics-Based Thresholding

TL;DR

This study proposes a federated threshold calculation method that aggregates summary statistics from normal and anomalous data across clients to create a global threshold for Anomaly Detection with federated Autoencoders, enhancing detection accuracy and robustness while ensuring privacy.

Abstract

In Federated Learning (FL), anomaly detection (AD) is a challenging task due to the decentralized nature of data and the presence of non-IID data distributions. This study introduces a novel federated threshold calculation method that leverages summary statistics from both normal and anomalous data to improve the accuracy and robustness of anomaly detection using autoencoders (AE) in a federated setting. Our approach aggregates local summary statistics across clients to compute a global threshold that optimally separates anomalies from normal data while ensuring privacy preservation. We conducted extensive experiments using publicly available datasets, including Credit Card Fraud Detection, Shuttle, and Covertype, under various data distribution scenarios. The results demonstrate that our method consistently outperforms existing federated and local threshold calculation techniques, particularly in handling non-IID data distributions. This study also explores the impact of different data distribution scenarios and the number of clients on the performance of federated anomaly detection. Our findings highlight the potential of using summary statistics for threshold calculation in improving the scalability and accuracy of federated anomaly detection systems.

Paper Structure

This paper contains 26 sections, 7 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Problem Definition
  4. Method
  5. Results
  6. Conclusion & Future Work
  7. Data Availability

Figures (7)

  • Figure 1: Federated Threshold Calculation using Summary Statistics
  • Figure 2: Comparison of F1 Scores Across Different Threshold Calculation Methods Using Randomly Distributed Data
  • Figure 3: Visualization of Error Distribution with Overlap Regions Defined by Upper and Lower Bounds Across Different Datasets and Data Distributions
  • Figure 4: Impact of Number of Clients on F1 Scores Across Different Threshold Calculation Methods and Datasets
  • Figure 5: Execution time of different threshold calculation methods
  • ...and 2 more figures