Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SOUL: A Semi-supervised Open-world continUal Learning method for Network Intrusion Detection

Suresh Kumar Amalapuram, Shreya Kumar, Bheemarjuna Reddy Tamma, Sumohana Channappayya

TL;DR

This work tackles the challenge of detecting novel cyber attacks in network intrusion data when labeled examples are scarce and test distributions shift over time. It introduces SOUL, a semi-supervised open-world continual learning framework that combines buffer memory with gradient projection memory (GPM) and confidence-based pseudo-labeling to mitigate catastrophic forgetting and enable high-confidence labeling of unseen tasks. Through extensive experiments on four public NID datasets, SOUL matches or closely approaches fully supervised baselines using as little as 20% labeled data and achieves substantial labeling-effort savings (roughly 11–46% across unseen tasks). The approach demonstrates robust minority-class performance improvements and provides practical mechanisms (agreement steps, cosine-distance voting, and analyst-in-the-loop labeling) to manage open-world data with limited supervision, offering a scalable path for real-world NID deployment.

Abstract

Fully supervised continual learning methods have shown improved attack traffic detection in a closed-world learning setting. However, obtaining fully annotated data is an arduous task in the security domain. Further, our research finds that after training a classifier on two days of network traffic, the performance decay of attack class detection over time (computed using the area under the time on precision-recall AUC of the attack class) drops from 0.985 to 0.506 on testing with three days of new test samples. In this work, we focus on label scarcity and open-world learning (OWL) settings to improve the attack class detection of the continual learning-based network intrusion detection (NID). We formulate OWL for NID as a semi-supervised continual learning-based method, dubbed SOUL, to achieve the classifier performance on par with fully supervised models while using limited annotated data. The proposed method is motivated by our empirical observation that using gradient projection memory (constructed using buffer memory samples) can significantly improve the detection performance of the attack (minority) class when trained using partially labeled data. Further, using the classifier's confidence in conjunction with buffer memory, SOUL generates high-confidence labels whenever it encounters OWL tasks closer to seen tasks, thus acting as a label generator. Interestingly, SOUL efficiently utilizes samples in the buffer memory for sample replay to avoid catastrophic forgetting, construct the projection memory, and assist in generating labels for unseen tasks. The proposed method is evaluated on four standard network intrusion detection datasets, and the performance results are closer to the fully supervised baselines using at most 20% labeled data while reducing the data annotation effort in the range of 11 to 45% for unseen data.

SOUL: A Semi-supervised Open-world continUal Learning method for Network Intrusion Detection

TL;DR

This work tackles the challenge of detecting novel cyber attacks in network intrusion data when labeled examples are scarce and test distributions shift over time. It introduces SOUL, a semi-supervised open-world continual learning framework that combines buffer memory with gradient projection memory (GPM) and confidence-based pseudo-labeling to mitigate catastrophic forgetting and enable high-confidence labeling of unseen tasks. Through extensive experiments on four public NID datasets, SOUL matches or closely approaches fully supervised baselines using as little as 20% labeled data and achieves substantial labeling-effort savings (roughly 11–46% across unseen tasks). The approach demonstrates robust minority-class performance improvements and provides practical mechanisms (agreement steps, cosine-distance voting, and analyst-in-the-loop labeling) to manage open-world data with limited supervision, offering a scalable path for real-world NID deployment.

Abstract

Fully supervised continual learning methods have shown improved attack traffic detection in a closed-world learning setting. However, obtaining fully annotated data is an arduous task in the security domain. Further, our research finds that after training a classifier on two days of network traffic, the performance decay of attack class detection over time (computed using the area under the time on precision-recall AUC of the attack class) drops from 0.985 to 0.506 on testing with three days of new test samples. In this work, we focus on label scarcity and open-world learning (OWL) settings to improve the attack class detection of the continual learning-based network intrusion detection (NID). We formulate OWL for NID as a semi-supervised continual learning-based method, dubbed SOUL, to achieve the classifier performance on par with fully supervised models while using limited annotated data. The proposed method is motivated by our empirical observation that using gradient projection memory (constructed using buffer memory samples) can significantly improve the detection performance of the attack (minority) class when trained using partially labeled data. Further, using the classifier's confidence in conjunction with buffer memory, SOUL generates high-confidence labels whenever it encounters OWL tasks closer to seen tasks, thus acting as a label generator. Interestingly, SOUL efficiently utilizes samples in the buffer memory for sample replay to avoid catastrophic forgetting, construct the projection memory, and assist in generating labels for unseen tasks. The proposed method is evaluated on four standard network intrusion detection datasets, and the performance results are closer to the fully supervised baselines using at most 20% labeled data while reducing the data annotation effort in the range of 11 to 45% for unseen data.

Paper Structure

This paper contains 15 sections, 10 equations, 3 figures, 10 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Semi-Supervised Continual Learning for NID
  5. Methodology
  6. Threat model and assumptions
  7. Leveraging orthogonal projections as complementary inductive biases: An empirical motivation
  8. Training scheme
  9. Training with seen tasks
  10. Training with unseen tasks in OWL
  11. Pseudo-code of the proposed SOUL method
  12. Experiments and Analysis
  13. Ablation studies
  14. Details of the Hyperparameters
  15. Summary

Figures (3)

  • Figure 1: Graphical illustration of the end-to-end training process of the proposed SOUL method on each of the seen tasks.
  • Figure 2: Graphical illustration of the end-to-end training process of the proposed SOUL method on each unseen task in open-world learning.
  • Figure 3: Demonstrating the sensitivity of the cosine distance and majority voting scheme to the percentage of labeling effort saved and the detection performance of the unseen attack class (unseen-AUT (A)). The percentage of savings in the labeling effort is displayed above the respective bar plot and marked in green.