Attack-Specialized Deep Learning with Ensemble Fusion for Network Anomaly Detection
Nisith Dissanayake, Uthayasanker Thayasivam
TL;DR
This work tackles the challenge of detecting diverse and imbalanced network intrusions in the NSL-KDD dataset by proposing an attack-specialized deep ensemble. Four binary detectors (DoS, Probe, R2L, U2R) are trained with tailored architectures and imbalance-handling techniques, and their outputs are fused by a Random Forest meta-classifier to produce the final decision. The approach leverages CNN-LSTM variants with attention, SMOTE oversampling, focal losses, threshold optimization, and cost-sensitive learning to address rare classes, achieving high per-class performance and near-perfect anomaly detection on benchmark splits. Empirical results show superior accuracy and recall across attack types and strong generalization across distribution shifts, underscoring the practical potential of modular, specialized ensembles for robust NIDS deployment.
Abstract
The growing scale and sophistication of cyberattacks pose critical challenges to network security, particularly in detecting diverse intrusion types within imbalanced datasets. Traditional intrusion detection systems (IDS) often struggle to maintain high accuracy across both frequent and rare attacks, leading to increased false negatives for minority classes. To address this, we propose a hybrid anomaly detection framework that integrates specialized deep learning models with an ensemble meta-classifier. Each model is trained to detect a specific attack category, enabling tailored learning of class-specific patterns, while their collective outputs are fused by a Random Forest meta-classifier to improve overall decision reliability. The framework is evaluated on the NSL-KDD benchmark, demonstrating superior performance in handling class imbalance compared to conventional monolithic models. Results show significant improvements in precision, recall, and F1-score across all attack categories, including rare classes such as User to Root (U2R). The proposed system achieves near-perfect detection rates with minimal false alarms, highlighting its robustness and generalizability. This work advances the design of intrusion detection systems by combining specialization with ensemble learning, providing an effective and scalable solution for safeguarding modern networks.
