Self-Supervised Transformer-based Contrastive Learning for Intrusion Detection Systems

TL;DR

Zero-day intrusion threats motivate learning from unlabeled traffic to generalize beyond signatures.The paper introduces a transformer encoder trained with a contrastive self-supervised objective on raw packet sequences, using a simple packet-replacement augmentation to create positive views.Evaluations across multiple datasets show improved intra- and inter-dataset anomaly detection and robust few-shot supervised finetuning, with up to 20% inter-dataset AUC gains and up to 2.5% few-shot gains.The approach demonstrates strong cross-domain generalization and practicality for real-time IDS that operate directly on packet data rather than NetFlow statistics.

Abstract

As the digital landscape becomes more interconnected, the frequency and severity of zero-day attacks, have significantly increased, leading to an urgent need for innovative Intrusion Detection Systems (IDS). Machine Learning-based IDS that learn from the network traffic characteristics and can discern attack patterns from benign traffic offer an advanced solution to traditional signature-based IDS. However, they heavily rely on labeled datasets, and their ability to generalize when encountering unseen traffic patterns remains a challenge. This paper proposes a novel self-supervised contrastive learning approach based on transformer encoders, specifically tailored for generalizable intrusion detection on raw packet sequences. Our proposed learning scheme employs a packet-level data augmentation strategy combined with a transformer-based architecture to extract and generate meaningful representations of traffic flows. Unlike traditional methods reliant on handcrafted statistical features (NetFlow), our approach automatically learns comprehensive packet sequence representations, significantly enhancing performance in anomaly identification tasks and supervised learning for intrusion detection. Our transformer-based framework exhibits better performance in comparison to existing NetFlow self-supervised methods. Specifically, we achieve up to a 3% higher AUC in anomaly detection for intra-dataset evaluation and up to 20% higher AUC scores in inter-dataset evaluation. Moreover, our model provides a strong baseline for supervised intrusion detection with limited labeled data, exhibiting an improvement over self-supervised NetFlow models of up to 1.5% AUC when pretrained and evaluated on the same dataset. Additionally, we show the adaptability of our pretrained model when fine-tuned across different datasets, demonstrating strong performance even when lacking benign data from the target domain.

Figures (2)

• Figure 1: Overall architecture of the transformer-based model
• Figure 2: Overview of the augmentation and contrastive learning procedure