Anomaly Detection Based on Critical Paths for Deep Neural Networks

TL;DR

The paper tackles reliable anomaly detection for deep neural networks by proposing ADCP, which extracts a small set of critical detection paths via genetic evolution and employs a random-subspace ensemble with SVDD scoring and class-wise thresholds. It demonstrates that using multiple, class-specific paths and voting substantially improves detection across AD, OOD, and NS types compared to state-of-the-art detectors and other path-extraction methods. The work provides extensive empirical evidence across MNIST, CIFAR-10, and SVHN, showing robustness to attack variety and cross-type generalization, while also analyzing hyper-parameter settings and path contributions. The approach offers a principled, interpretable path-based view of model decision logic that enhances practical detection capabilities in safety-critical settings, albeit at notable computational cost that the authors acknowledge and discuss for future work.

Abstract

Deep neural networks (DNNs) are notoriously hard to understand and difficult to defend. Extracting representative paths (including the neuron activation values and the connections between neurons) from DNNs using software engineering approaches has recently shown to be a promising approach in interpreting the decision making process of blackbox DNNs, as the extracted paths are often effective in capturing essential features. With this in mind, this work investigates a novel approach that extracts critical paths from DNNs and subsequently applies the extracted paths for the anomaly detection task, based on the observation that outliers and adversarial inputs do not usually induce the same activation pattern on those paths as normal (in-distribution) inputs. In our approach, we first identify critical detection paths via genetic evolution and mutation. Since different paths in a DNN often capture different features for the same target class, we ensemble detection results from multiple paths by integrating random subspace sampling and a voting mechanism. Compared with state-of-the-art methods, our experimental results suggest that our method not only outperforms them, but it is also suitable for the detection of a broad range of anomaly types with high accuracy.

Figures (10)

• Figure 1: Two-dimensional representations of features extracted from the critical detection path of a LeNet model trained on MNIST. The feature clusters for the $10$ classes are shown, with green dots for MNIST (normal) data and brown dots for F-MNIST (anomaly) data.
• Figure 2: The overview of our approach for selecting critical detection path.
• Figure 3: The AUROC corresponding to different paths on five models for different anomaly samples.
• Figure 4: The TPR corresponding to the mutation number of ADCP on different models for adversarial attack in each class.
• Figure 5: The TPR corresponding to the mutation number of ADCP on different models for OOD in each class.

Theorems & Definitions (1)

• definition 1