RangeAD: Fast On-Model Anomaly Detection

Abstract

In practice, machine learning methods commonly require anomaly detection (AD) to filter inputs or detect distributional shifts. Typically, this is implemented by running a separate AD model alongside the primary model. However, this separation ignores the fact that the primary model already encodes substantial information about the target distribution. In this paper, we introduce On-Model AD, a setting for anomaly detection that explicitly leverages access to a related machine learning model. Within this setting, we propose RangeAD, an algorithm that utilizes neuron-wise output ranges derived from the primary model. RangeAD achieves superior performance even on high-dimensional tasks while incurring substantially lower inference costs. Our results demonstrate the potential of the On-Model AD setting as a practical framework for efficient anomaly detection.

Figures (5)

• Figure 1: Depiction of our proposed methodology. (a) represents the neural network, from which the sample activations per neuron are depicted in (b). From these distributions - to be considered normal - interval borders are derived. The activations produced for a new test input are then checked per neuron to see whether it falls inside the corresponding interval in (c). For every test sample $x_i$the amount of feature-outputs lying outside of the corresponding interval indicates the anomaly grade of $x_i$.
• Figure 2: AUC-ROC of different anomaly detection methods over 12 tabular datasets. Our method is run three times with models having hidden-layer neuron counts of 100, 500, and 1000. It reaches the highest AUC-ROC on average compared to all competitors.
• Figure 3: AUC-ROC and inference time of different anomaly detection methods averaged over 12 tabular datasets. Our method is run three times with models with hidden layer neuron counts of 100, 500, 1000. It achieves the highest detection performance while providing the fastest inference runtime of all competitors. We provide the same analysis on the training time in the supplementary material.
• Figure 4: When using differently sized hidden layers in our neural networks, AUC ROC and model performance are correlated. The results represent the average of 5 runs with a standard deviation over the 12 tabular datasets.
• Figure 5: Ablation study results.