A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks

TL;DR

The paper tackles detecting abnormal test samples—both out-of-distribution and adversarial inputs—for pretrained softmax classifiers. It introduces a simple generative framework that fits class-conditional Gaussians with a shared covariance on DNN features and uses a Mahalanobis distance-based score for detection, aligned with the softmax decision boundary via Gaussian discriminant analysis. It further enhances calibration with input pre-processing and multi-layer feature ensembles, and demonstrates applicability to class-incremental learning without retraining. Empirically, it achieves state-of-the-art or competitive performance across OOD and adversarial benchmarks and shows robustness to noisy labels and small datasets.

Abstract

Detecting test samples drawn sufficiently far away from the training distribution statistically or adversarially is a fundamental requirement for deploying a good classifier in many real-world machine learning applications. However, deep neural networks with the softmax classifier are known to produce highly overconfident posterior distributions even for such abnormal samples. In this paper, we propose a simple yet effective method for detecting any abnormal samples, which is applicable to any pre-trained softmax neural classifier. We obtain the class conditional Gaussian distributions with respect to (low- and upper-level) features of the deep models under Gaussian discriminant analysis, which result in a confidence score based on the Mahalanobis distance. While most prior methods have been evaluated for detecting either out-of-distribution or adversarial samples, but not both, the proposed method achieves the state-of-the-art performances for both cases in our experiments. Moreover, we found that our proposed method is more robust in harsh cases, e.g., when the training dataset has noisy labels or small number of samples. Finally, we show that the proposed method enjoys broader usage by applying it to class-incremental learning: whenever out-of-distribution samples are detected, our classification rule can incorporate new classes well without further training deep models.

Figures (11)

• Figure 1: Experimental results under the ResNet with 34 layers. (a) Visualization of final features from ResNet trained on CIFAR-10 by t-SNE, where the colors of points indicate the classes of the corresponding objects. (b) Classification test set accuracy of ResNet on CIFAR-10, CIFAR-100 and SVHN datasets. (c) Receiver operating characteristic (ROC) curves: the x-axis and y-axis represent the false positive rate (FPR) and true positive rate (TPR), respectively.
• Figure 2: AUROC (%) of threshold-based detector using the confidence score in \ref{['eq:mahalanobis']} computed at different basic blocks of DenseNet trained on CIFAR-10 dataset. We measure the detection performance using (a) TinyImageNet, (b) LSUN, (c) SVHN and (d) adversarial (DeepFool) samples.
• Figure 3: Comparison of AUROC (%) under extreme scenarios: (a) small number of training data, where the x-axis represents the number of training data. (b) Random label is assigned to training data, where the x-axis represents the percentage of training data with random label.
• Figure 4: Experimental results of class-incremental learning on CIFAR-100 and ImageNet datasets. In each experiment, we report (left) AUC with respect to the number of learned classes and, (right) the base-new class accuracy curve after the last new classes is added.
• Figure 5: Comparison of AUROC (%) under different training data. To evaluate the robustness of proposed method, we train ResNet (a) by varying the number of training data and (b) assigning random label to training data on CIFAR-10 dataset.