Latent Sculpting for Zero-Shot Generalization: A Manifold Learning Approach to Out-of-Distribution Anomaly Detection

TL;DR

This work tackles generalization collapse in high-dimensional tabular anomaly detection by introducing Latent Sculpting, a two-stage framework that first structures the latent space and then learns a precise density with a Masked Autoregressive Flow. Stage 1 uses a hybrid 1D-CNN and Transformer encoder trained with the Dual-Centroid Compactness Loss to sculpt benign traffic into a tight, hyperspherical manifold, separating it from known anomalies. Stage 2 conditions a MAF on this manifold to model its density, enabling robust zero-shot OOD detection via dynamic NLL thresholds. On CIC-IDS-2017, the method achieves high internal validation performance (e.g., F1 ~ 0.87, AUROC ~ 0.967) and dramatically improves zero-shot detection (e.g., 0.85 recall for unseen attacks and 88.89% for Infiltration), outperforming both supervised and unsupervised baselines while using far less labeled benign data. These results suggest that decoupling structure learning from density estimation yields a scalable, data-efficient path toward generalized anomaly detection in security and beyond.

Abstract

A fundamental limitation of supervised deep learning in high-dimensional tabular domains is "Generalization Collapse": models learn precise decision boundaries for known distributions but fail catastrophically when facing Out-of-Distribution (OOD) data. We hypothesize that this failure stems from the lack of topological constraints in the latent space, resulting in diffuse manifolds where novel anomalies remain statistically indistinguishable from benign data. To address this, we propose Latent Sculpting, a hierarchical two-stage representation learning framework. Stage 1 utilizes a hybrid 1D-CNN and Transformer Encoder trained with a novel Dual-Centroid Compactness Loss (DCCL) to actively "sculpt" benign traffic into a low-entropy, hyperspherical cluster. Unlike standard contrastive losses that rely on triplet mining, DCCL optimizes global cluster centroids to enforce absolute manifold density. Stage 2 conditions a Masked Autoregressive Flow (MAF) on this pre-structured manifold to learn an exact density estimate. We evaluate this methodology on the rigorous CIC-IDS-2017 benchmark, treating it as a proxy for complex, non-stationary data streams. Empirical results demonstrate that explicit manifold sculpting is a prerequisite for robust zero-shot generalization. While supervised baselines suffered catastrophic performance collapse on unseen distribution shifts (F1 approx 0.30) and the strongest unsupervised baseline achieved only 0.76, our framework achieved an F1-Score of 0.87 on strictly zero-shot anomalies. Notably, we report an 88.89% detection rate on "Infiltration" scenarios--a complex distributional shift where state-of-the-art supervised models achieved 0.00% accuracy. These findings suggest that decoupling structure learning from density estimation provides a scalable path toward generalized anomaly detection.

Figures (3)

• Figure 1: 1D-CNN Feature Extractor Architecture. The model accepts an input of shape $(B, 71)$, reshapes it, and processes it through five 1D convolutional layers with kernel size $k=2$. ReLU activations provide non-linearity, while Dropout ($p=0.1$) and MaxPool operations ensure robustness and dimensionality reduction before passing features to the Transformer stage.
• Figure 2: Transformer Encoder and Latent Projection Head. The feature sequence $(B, 16, 33)$ extracted by the CNN is permuted and projected to the Transformer embedding dimension $d_{model}=64$. After adding positional embeddings, the sequence is processed by a Transformer Encoder layer utilizing Multi-Head Attention and a Feed-Forward Network. Finally, Global Average Pooling aggregates the temporal dimension, and an MLP head projects the result to the final latent vector $z \in \mathbb{R}^{32}$.
• Figure 3: Masked Autoregressive Flow (MAF) Architecture. The density estimation stage consists of a stack of 16 MAF layers. Each layer receives the latent vector $z_{enc}$ from the Transformer Encoder (Stage 1) and transforms it through a Masked Autoencoder for Distribution Estimation (MADE). The MADE network outputs scale ($s$) and shift ($t$) parameters, which are used to apply an affine transformation $u = z \odot \exp(\tanh(s)) + t$. A dimension permutation (Flip) is applied between layers to capture dependencies in all directions.