Fog Intelligence for Network Anomaly Detection

TL;DR

The paper addresses the challenge of detecting anomalies in massive, heterogeneous 5G networks where centralized ML is impractical. It introduces fog intelligence, a distributed ML architecture that leverages both cloud and edge resources to enable scalable, privacy-preserving intelligent network management. It surveys distributed ML platforms and presents an example Deep Network Analyzer (DNA) for cell-level anomaly detection, including fingerprint learning via rare association rule mining and real-time root-cause analysis, implemented on Spark and adaptable to cloud or near-RNC deployments. The work demonstrates the feasibility and benefits of edge-to-cloud ML for 5G SQM, with potential improvements in QoS/QoE and reductions in data bandwidth and latency.

Abstract

Anomalies are common in network system monitoring. When manifested as network threats to be mitigated, service outages to be prevented, and security risks to be ameliorated, detecting such anomalous network behaviors becomes of great importance. However, the growing scale and complexity of the mobile communication networks, as well as the ever-increasing amount and dimensionality of the network surveillance data, make it extremely difficult to monitor a mobile network and discover abnormal network behaviors. Recent advances in machine learning allow for obtaining near-optimal solutions to complicated decision-making problems with many sources of uncertainty that cannot be accurately characterized by traditional mathematical models. However, most machine learning algorithms are centralized, which renders them inapplicable to a large-scale distributed wireless networks with tens of millions of mobile devices. In this article, we present fog intelligence, a distributed machine learning architecture that enables intelligent wireless network management. It preserves the advantage of both edge processing and centralized cloud computing. In addition, the proposed architecture is scalable, privacy-preserving, and well suited for intelligent management of a distributed wireless network.