SoK: Decoding the Enigma of Encrypted Network Traffic Classifiers

TL;DR

This work addresses the fragility of ML-based network traffic classifiers under modern TLS 1.3 encryption by systematizing design choices and benchmarking practices, and by introducing CipherSpectrum to replace outdated datasets. Through 348 occlusion experiments across state-of-the-art models, it demonstrates how dataset age, design oversights, and unsubstantiated assumptions drive overfitting and misleading conclusions. The study provides actionable guidelines to avoid data leakage, contextual and temporal overfitting, and to evaluate encrypted payloads and headers more responsibly. Overall, CipherSpectrum and the empirical findings push toward more robust, generalizable NTC methods suitable for real-world encrypted traffic.

Abstract

The adoption of modern encryption protocols such as TLS 1.3 has significantly challenged traditional network traffic classification (NTC) methods. As a consequence, researchers are increasingly turning to machine learning (ML) approaches to overcome these obstacles. In this paper, we comprehensively analyze ML-based NTC studies, developing a taxonomy of their design choices, benchmarking suites, and prevalent assumptions impacting classifier performance. Through this systematization, we demonstrate widespread reliance on outdated datasets, oversights in design choices, and the consequences of unsubstantiated assumptions. Our evaluation reveals that the majority of proposed encrypted traffic classifiers have mistakenly utilized unencrypted traffic due to the use of legacy datasets. Furthermore, by conducting 348 feature occlusion experiments on state-of-the-art classifiers, we show how oversights in NTC design choices lead to overfitting, and validate or refute prevailing assumptions with empirical evidence. By highlighting lessons learned, we offer strategic insights, identify emerging research directions, and recommend best practices to support the development of real-world applicable NTC methodologies.

Figures (2)

• Figure 1: Taxonomy of design choices in raw information based NTC
• Figure 2: Public network traffic dataset evaluation.