Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Network Anomaly Detection in Distributed Edge Computing Infrastructure

William Marfo, Enrique A. Rico, Deepak K. Tosh, Shirley V. Moore

TL;DR

It is shown that by leveraging distributed computing and containerization technologies, the framework not only improves scalability and fault tolerance but also achieves superior detection performance compared to state-of-the-art methods.

Abstract

As networks continue to grow in complexity and scale, detecting anomalies has become increasingly challenging, particularly in diverse and geographically dispersed environments. Traditional approaches often struggle with managing the computational burden associated with analyzing large-scale network traffic to identify anomalies. This paper introduces a distributed edge computing framework that integrates federated learning with Apache Spark and Kubernetes to address these challenges. We hypothesize that our approach, which enables collaborative model training across distributed nodes, significantly enhances the detection accuracy of network anomalies across different network types. By leveraging distributed computing and containerization technologies, our framework not only improves scalability and fault tolerance but also achieves superior detection performance compared to state-of-the-art methods. Extensive experiments on the UNSW-NB15 and ROAD datasets validate the effectiveness of our approach, demonstrating statistically significant improvements in detection accuracy and training efficiency over baseline models, as confirmed by Mann-Whitney U and Kolmogorov-Smirnov tests (p < 0.05).

Network Anomaly Detection in Distributed Edge Computing Infrastructure

TL;DR

It is shown that by leveraging distributed computing and containerization technologies, the framework not only improves scalability and fault tolerance but also achieves superior detection performance compared to state-of-the-art methods.

Abstract

As networks continue to grow in complexity and scale, detecting anomalies has become increasingly challenging, particularly in diverse and geographically dispersed environments. Traditional approaches often struggle with managing the computational burden associated with analyzing large-scale network traffic to identify anomalies. This paper introduces a distributed edge computing framework that integrates federated learning with Apache Spark and Kubernetes to address these challenges. We hypothesize that our approach, which enables collaborative model training across distributed nodes, significantly enhances the detection accuracy of network anomalies across different network types. By leveraging distributed computing and containerization technologies, our framework not only improves scalability and fault tolerance but also achieves superior detection performance compared to state-of-the-art methods. Extensive experiments on the UNSW-NB15 and ROAD datasets validate the effectiveness of our approach, demonstrating statistically significant improvements in detection accuracy and training efficiency over baseline models, as confirmed by Mann-Whitney U and Kolmogorov-Smirnov tests (p < 0.05).

Paper Structure

This paper contains 25 sections, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Federated Learning
  4. Application in Network Anomaly Detection
  5. Related Work
  6. Proposed Federated Learning Framework for Network Anomaly Detection
  7. System Architecture
  8. Federated Learning Architecture for Network Anomaly Detection
  9. Recovery protocol without checkpointing
  10. Recovery protocol with checkpointing
  11. Optimal checkpointing interval
  12. Evaluation Results
  13. Experimental Setup
  14. Datasets
  15. UNSW-NB15 dataset
  16. ...and 10 more sections

Figures (3)

  • Figure 1: FL architecture for network anomaly detection, illustrating model selection, training, checkpointing, aggregation, and evaluation.
  • Figure 2: Training performance of models in terms of loss and accuracy over 300 epochs on the UNSW-NB15 and ROAD datasets.
  • Figure 3: Performance comparison on the UNSW-NB15 dataset (top row) and the ROAD dataset (bottom row). The left column shows accuracy as a function of the number of clients, and the right column shows accuracy under different dropout rates.