Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Take Package as Language: Anomaly Detection Using Transformer

Jie Huang

TL;DR

This paper introduces NIDS-GPT, a GPT-based causal language model for network intrusion detection that treats every numeric token in a packet as an independent word and uses a specialized tokenizer and multi-faceted embeddings to capture packet structure. By predicting the next token and appending the packet label as the final token, the model reframes intrusion detection as a language modeling task, enabling unsupervised pre-training, scalable learning, and interpretable attention visualization. Empirical results on CICIDS2017 and car-hacking CAN datasets show exceptional performance under extreme data imbalance and strong one-shot learning capabilities, with demonstrated transferability to new network scenarios. The work offers a promising, end-to-end framework for robust, interpretable NIDS in data-scarce and resource-constrained environments.

Abstract

Network data packet anomaly detection faces numerous challenges, including exploring new anomaly supervision signals, researching weakly supervised anomaly detection, and improving model interpretability. This paper proposes NIDS-GPT, a GPT-based causal language model for network intrusion detection. Unlike previous work, NIDS-GPT innovatively treats each number in the packet as an independent "word" rather than packet fields, enabling a more fine-grained data representation. We adopt an improved GPT-2 model and design special tokenizers and embedding layers to better capture the structure and semantics of network data. NIDS-GPT has good scalability, supports unsupervised pre-training, and enhances model interpretability through attention weight visualization. Experiments on the CICIDS2017 and car-hacking datasets show that NIDS-GPT achieves 100\% accuracy under extreme imbalance conditions, far surpassing traditional methods; it also achieves over 90\% accuracy in one-shot learning. These results demonstrate NIDS-GPT's excellent performance and potential in handling complex network anomaly detection tasks, especially in data-imbalanced and resource-constrained scenarios. The code is available at \url{https://github.com/woshixiaobai2019/nids-gpt.gi

Take Package as Language: Anomaly Detection Using Transformer

TL;DR

This paper introduces NIDS-GPT, a GPT-based causal language model for network intrusion detection that treats every numeric token in a packet as an independent word and uses a specialized tokenizer and multi-faceted embeddings to capture packet structure. By predicting the next token and appending the packet label as the final token, the model reframes intrusion detection as a language modeling task, enabling unsupervised pre-training, scalable learning, and interpretable attention visualization. Empirical results on CICIDS2017 and car-hacking CAN datasets show exceptional performance under extreme data imbalance and strong one-shot learning capabilities, with demonstrated transferability to new network scenarios. The work offers a promising, end-to-end framework for robust, interpretable NIDS in data-scarce and resource-constrained environments.

Abstract

Network data packet anomaly detection faces numerous challenges, including exploring new anomaly supervision signals, researching weakly supervised anomaly detection, and improving model interpretability. This paper proposes NIDS-GPT, a GPT-based causal language model for network intrusion detection. Unlike previous work, NIDS-GPT innovatively treats each number in the packet as an independent "word" rather than packet fields, enabling a more fine-grained data representation. We adopt an improved GPT-2 model and design special tokenizers and embedding layers to better capture the structure and semantics of network data. NIDS-GPT has good scalability, supports unsupervised pre-training, and enhances model interpretability through attention weight visualization. Experiments on the CICIDS2017 and car-hacking datasets show that NIDS-GPT achieves 100\% accuracy under extreme imbalance conditions, far surpassing traditional methods; it also achieves over 90\% accuracy in one-shot learning. These results demonstrate NIDS-GPT's excellent performance and potential in handling complex network anomaly detection tasks, especially in data-imbalanced and resource-constrained scenarios. The code is available at \url{https://github.com/woshixiaobai2019/nids-gpt.gi

Paper Structure

This paper contains 16 sections, 14 equations, 6 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Overview
  5. Network Packet Tokenizer
  6. Network Packet Embedding
  7. Objective Function and Inference
  8. Evaluation Metrics
  9. Precision
  10. Recall
  11. F1 Score
  12. Experiments
  13. Network Traffic Packet Detection Experiment
  14. Vehicle Network Data Packet Detection Experiment
  15. Interpretability Experiment
  16. ...and 1 more sections

Figures (6)

  • Figure 1: The packet language
  • Figure 2: Comparison of our method with existing methods
  • Figure 3: Overall model design
  • Figure 4: Training log
  • Figure 5: Network anomaly detection attention visualization
  • ...and 1 more figures