FastFlow: Early Yet Robust Network Flow Classification using the Minimal Number of Time-Series Packets

TL;DR

FastFlow addresses the need for early, accurate network-flow classification at scale by introducing a dual-grained time-series representation (packet-level and slot-level) and RL-trained LSTM classifiers that dynamically determine the minimal data length required per flow. By training with reinforcement learning and augmenting data to simulate unknown flows, FastFlow achieves robust open-world detection and resilience to packet sequence disorders. The method demonstrates strong performance on public datasets and a large campus deployment, achieving over 91% accuracy with an average of 8.37 initial packets and around 0.5 seconds per flow, and effectively distinguishing unknown flows. This approach enables ISP-scale flow labeling for applications and content providers with practical efficiency and accuracy, facilitating real-time network monitoring and optimization.

Abstract

Network traffic classification is of great importance for network operators in their daily routines, such as analyzing the usage patterns of multimedia applications and optimizing network configurations. Internet service providers (ISPs) that operate high-speed links expect network flow classifiers to accurately classify flows early, using the minimal number of necessary initial packets per flow. These classifiers must also be robust to packet sequence disorders in candidate flows and capable of detecting unseen flow types that are not within the existing classification scope, which are not well achieved by existing methods. In this paper, we develop FastFlow, a time-series flow classification method that accurately classifies network flows as one of the known types or the unknown type, which dynamically selects the minimal number of packets to balance accuracy and efficiency. Toward the objectives, we first develop a flow representation process that converts packet streams at both per-packet and per-slot granularity for precise packet statistics with robustness to packet sequence disorders. Second, we develop a sequential decision-based classification model that leverages LSTM architecture trained with reinforcement learning. Our model makes dynamic decisions on the minimal number of time-series data points per flow for the confident classification as one of the known flow types or an unknown one. We evaluated our method on public datasets and demonstrated its superior performance in early and accurate flow classification. Deployment insights on the classification of over 22.9 million flows across seven application types and 33 content providers in a campus network over one week are discussed, showing that FastFlow requires an average of only 8.37 packets and 0.5 seconds to classify the application type of a flow with over 91% accuracy and over 96% accuracy for the content providers.

Figures (9)

• Figure 1: Overview of our FastFlow flow classification process for large networks.
• Figure 2: A candidate flow being represented as per packet and per slot time-series data sequence.
• Figure 3: The architecture of FastFlow time-series flow classifier realizing a sequential decision-making process.
• Figure 4: (a) Under the closed-world assumption, a classifier can naively split the entire feature space according to the existing classes. (b) The resulting classifier misclassifies unknown instances as belonging to one of the existing classes. (c) We use augmentation in classifier training to generate closely outlying synthetic unknown flows that help bind the classes' decision borders.
• Figure 5: Performance of classifiers when using time-series statistics from a fixed number of packets or time-interval slots on the three public datasets. The bounded range of each data point shows the variation of accuracy/marcoF1 across flow types in each dataset.