Traffic-MoE: A Sparse Foundation Model for Network Traffic Analysis

TL;DR

Traffic-MoE tackles the challenge of deploying high-capacity traffic-analysis models in real-time environments by employing a sparse Mixture-of-Experts backbone that routes tokens to a small set of specialized experts, effectively decoupling model capacity from inference cost. The framework combines Traffic2Token byte-level tokenization, causal masked self-attention, and a dual-path MoE with load-balancing, trained via autoregressive pre-training and few-shot fine-tuning on diverse traffic tasks. Empirical results show state-of-the-art performance across intrusion detection, service classification, and encrypted traffic analysis tasks, with substantial gains in throughput and reductions in latency and GPU memory, as well as strong robustness to distribution shifts and adversarial shaping. This approach demonstrates a scalable, efficient path to operationally deployable foundation models for real-time network defense, enabling robust detection in encrypted and obfuscated traffic while maintaining high throughput.

Abstract

While pre-trained large models have achieved state-of-the-art performance in network traffic analysis, their prohibitive computational costs hinder deployment in real-time, throughput-sensitive network defense environments. This work bridges the gap between advanced representation learning and practical network protection by introducing Traffic-MoE, a sparse foundation model optimized for high-efficiency real-time inference. By dynamically routing traffic tokens to a small subset of specialized experts, Traffic-MoE effectively decouples model capacity from computational overhead. Extensive evaluations across three security-oriented tasks demonstrate that Traffic-MoE achieves up to a 12.38% improvement in detection performance compared to leading dense competitors. Crucially, it delivers a 91.62% increase in throughput, reduces inference latency by 47.81%, and cuts peak GPU memory consumption by 38.72%. Beyond efficiency, Traffic-MoE exhibits superior robustness against adversarial traffic shaping and maintains high detection efficacy in few-shot scenarios, establishing a new paradigm for scalable and resilient network traffic analysis.

Figures (13)

• Figure 1: Threat model and operational scenario.
• Figure 2: The overall framework of Traffic-MoE. It consists of the Traffic2Token serialization module, the sparse Mixture-of-Experts (MoE) backbone, and the pre-training/fine-tuning pipeline for downstream security tasks.
• Figure 3: Illustration of flow token sequence representation.
• Figure 4: Illustration of Traffic-MoE backbone architecture.
• Figure 5: Performance confusion matrices of Traffic-MoE across different service classification tasks.