Hybrid Quantum-Classical Autoencoders for Unsupervised Network Intrusion Detection

TL;DR

This work provides the first large-scale, data-driven benchmark of hybrid quantum-classical autoencoders for unsupervised network intrusion detection, systematically varying quantum-layer placement, data encoding, readout, and regularization. It shows that well-tuned HQC models can match or exceed classical baselines and generalize better to zero-day attacks, though they are highly sensitive to architectural choices and susceptible to gate-noise. The study delivers concrete guidance—early quantum-layer integration and reconstruction-error-based anomaly scoring improve performance—and highlights practical challenges like noise sensitivity that must be addressed for near-term deployment. The authors also offer reproducible code and datasets, laying groundwork for further hardware-aware HQC-NIDS research.

Abstract

Unsupervised anomaly-based intrusion detection requires models that can generalize to attack patterns not observed during training. This work presents the first large-scale evaluation of hybrid quantum-classical (HQC) autoencoders for this task. We construct a unified experimental framework that iterates over key quantum design choices, including quantum-layer placement, measurement approach, variational and non-variational formulations, and latent-space regularization. Experiments across three benchmark NIDS datasets show that HQC autoencoders can match or exceed classical performance in their best configurations, although they exhibit higher sensitivity to architectural decisions. Under zero-day evaluation, well-configured HQC models provide stronger and more stable generalization than classical and supervised baselines. Simulated gate-noise experiments reveal early performance degradation, indicating the need for noise-aware HQC designs. These results provide the first data-driven characterization of HQC autoencoder behavior for network intrusion detection and outline key factors that govern their practical viability. All experiment code and configurations are available at https://github.com/arasyi/hqcae-network-intrusion-detection.

Figures (5)

• Figure 1: Unified HQC autoencoder architecture showing QLayer placement, internals, and anomaly detection mechanisms.
• Figure 2: Violin and swarm plots of AUROC distributions for all classical and HQC configurations across the three datasets.
• Figure 3: Violin and swarm plots of AUROC for models under the LOAO protocol.
• Figure 4: UMAP visualization of an autoencoder's latent space on the NSL-KDD test set.
• Figure 5: Impact of simulated coherent gate error on AUROC for the best HQC model on UNSW-NB15.