A Less Biased Evaluation of Out-of-distribution Sample Detectors

TL;DR

This work critiques the common practice of evaluating out-of-distribution detectors with only two datasets, introducing OD-test, a three-dataset benchmark that folds unseen outliers into the test phase to better reflect real-world uncertainty. By evaluating a broad set of methods on high-dimensional image tasks, the authors show that many approaches overfit to known outliers and lose reliability when faced with unknown anomalies, with overall accuracy often remaining below 80%. The study emphasizes that more conservative, robust evaluation is essential for deploying trustworthy OOD detectors and provides open-source code to promote reproducibility. The findings urge the community to adopt OD-test and pursue methods that generalize to unknown, diverse outliers rather than optimizing for known ones.

Abstract

In the real world, a learning system could receive an input that is unlike anything it has seen during training. Unfortunately, out-of-distribution samples can lead to unpredictable behaviour. We need to know whether any given input belongs to the population distribution of the training/evaluation data to prevent unpredictable behaviour in deployed systems. A recent surge of interest in this problem has led to the development of sophisticated techniques in the deep learning literature. However, due to the absence of a standard problem definition or an exhaustive evaluation, it is not evident if we can rely on these methods. What makes this problem different from a typical supervised learning setting is that the distribution of outliers used in training may not be the same as the distribution of outliers encountered in the application. Classical approaches that learn inliers vs. outliers with only two datasets can yield optimistic results. We introduce OD-test, a three-dataset evaluation scheme as a more reliable strategy to assess progress on this problem. We present an exhaustive evaluation of a broad set of methods from related areas on image classification tasks. Contrary to the existing results, we show that for realistic applications of high-dimensional images the previous techniques have low accuracy and are not reliable in practice.

Figures (7)

• Figure 1: The predictions of several popular networks Huang2017Iandola2016He2016Simonyan2014Krizhevsky2012 that are trained on ImageNet on unseen data. The red predictions are entirely wrong, the green predictions are justifiable, and the orange predictions are less justifiable. The middle image is Gaussian noise. We show that thresholding the output probability is not a reliable defence.
• Figure 3: Evaluation with two datasets versus OD-test. Evaluating OOD detectors with only two distributions can be misleading for practical applications. The error bars are the 95% confidence level. The two-dataset evaluations are over all possible pairs of datasets ($n=46$), whereas the OD-test evaluations are over all possible triplets ($n=308$).
• Figure 4: The average test accuracy of the OOD detection methods over $308$ experiments/method with 95% confidence level. /VGG or /Res indicates the backing network architecture. #-NN./ is the number of nearest neighbours. A random prediction would have an accuracy of $0.5$.
• Figure 5: The test accuracy over 50 experiments/bar with 95% confidence level.
• Figure 7: The average test accuracy over 50 experiments per bar. The error bars indicate the 95% confidence level. The figure is best viewed in color.