ResidualSketch: Enhancing Layer Efficiency and Error Reduction in Hierarchical Heavy Hitter Detection with ResNet Innovations

TL;DR

The paper tackles the challenge of fast and accurate hierarchical heavy hitter detection in high-speed IP traffic. It introduces ResidualSketch, a hybrid sketch that combines HH-based and APK-based approaches by integrating Residual Blocks at pivotal IP-hierarchy levels and applying Residual Connections to prevent duplicate counting and curb Gradual Error Diffusion (GED), all while reducing memory and update latency. The authors provide formal analysis of GED, space/time complexity, unbiasedness, and variance, and validate the method through extensive experiments on CAIDA, MAWI, Campus, and synthetic datasets, showing superior F1 scores and lower ARE under modest memory budgets. The work also demonstrates that a 2–3 level configuration with residual connections offers robust performance, and the implementation is open-sourced, enabling practical deployment for real-time network anomaly detection. The approach promises improved scalability and robustness for 1D and 2D hierarchical traffic analysis in production networks.

Abstract

In network management, swiftly and accurately identifying traffic anomalies, including Distributed Denial-of-Service (DDoS) attacks and unexpected network disruptions, is essential for network stability and security. Key to this process is the detection of Hierarchical Heavy Hitters (HHH), which significantly aids in the management of high-speed IP traffic. This study introduces ResidualSketch, a novel algorithm for HHH detection in hierarchical traffic analysis. ResidualSketch distinguishes itself by incorporating Residual Blocks and Residual Connections at crucial layers within the IP hierarchy, thus mitigating the Gradual Error Diffusion (GED) phenomenon in previous methods and reducing memory overhead while maintaining low update latency. Through comprehensive experiments on various datasets, we demonstrate that ResidualSketch outperforms existing state-of-the-art solutions in terms of accuracy and update speed across multiple layers of the network hierarchy. All related codes of ResidualSketch are open-source at GitHub.

Figures (14)

• Figure 1: F1 Score for HHH Detection Across Various Layers in the MAWI Dataset mawi Using Cocosketch (100KB)
• Figure 2: The binary tree represents the hierarchical flow IDs structure, such as srcIP or dstIP, aggregating from layer 0 to $d$, from fully specified IPs to fully generalized '*'. In all figures, dark mauve nodes denote HHs. In this figure, they are shown with $T$ values (threshold $10k$ here), are counted from the bottom up to form a set known as HHH. In a 2D scenario, there are $(d + 1)^2$ nodes and $2d + 1$ layers of aggregation for HHH determination.
• Figure 3: The Architecture of Our Algorithm (A*, A' represents A's ancestor)
• Figure 4: Insertion Examples of Residual Sketch (USS as the Residual Block)
• Figure 5: HH ratio of Different Layers on the CAIDA and MAWI Datasets

Theorems & Definitions (4)

• Definition 1
• Definition 2
• Theorem 1
• proof