Fast Decision Boundary based Out-of-Distribution Detector

TL;DR

This paper proposes a computationally-efficient OOD detector without using auxiliary models while still leveraging the rich information embedded in the feature space, and develops a hyperparameter-free, auxiliary model-free OOD detector.

Abstract

Efficient and effective Out-of-Distribution (OOD) detection is essential for the safe deployment of AI systems. Existing feature space methods, while effective, often incur significant computational overhead due to their reliance on auxiliary models built from training features. In this paper, we propose a computationally-efficient OOD detector without using auxiliary models while still leveraging the rich information embedded in the feature space. Specifically, we detect OOD samples based on their feature distances to decision boundaries. To minimize computational cost, we introduce an efficient closed-form estimation, analytically proven to tightly lower bound the distance. Based on our estimation, we discover that In-Distribution (ID) features tend to be further from decision boundaries than OOD features. Additionally, ID and OOD samples are better separated when compared at equal deviation levels from the mean of training features. By regularizing the distances to decision boundaries based on feature deviation from the mean, we develop a hyperparameter-free, auxiliary model-free OOD detector. Our method matches or surpasses the effectiveness of state-of-the-art methods in extensive experiments while incurring negligible overhead in inference latency. Overall, our approach significantly improves the efficiency-effectiveness trade-off in OOD detection. Code is available at: https://github.com/litianliu/fDBD-OOD.

Figures (7)

• Figure 1: Overview.Left: Conceptual Illustration. The feature distance to decision boundaries on a multi-class classifier's penultimate layer, quantifying the perturbation magnitude needed to alter the model prediction to a class (see formal definition in Section \ref{['sec:dist-measure']}). Right: Empirical Observation. Features of ID samples (CIFAR-10) tend to reside further from decision boundaries than OOD samples (SVHN). The distances are measured using our method (see Section \ref{['sec:dist-measure']}) and averages are per sample.
• Figure 2: Regularization enhances ID/OOD separation.Left: Histograms of ID/OOD features based on the average distance to decision boundaries. Right: Histograms of ID/OOD features based on the regularized average distance to decision boundaries, which effectively compares ID and OOD features at equal deviation levels from the mean of training features.
• Figure 3: ID and OOD are better separated at Equal Deviation Levels. Features are grouped by deviation levels with group mean and variance displayed. Since the average feature distance to decision boundaries increases as features deviate from the mean of training features, the circled ID/OOD groups cannot be distinguished based on their average distance to decision boundaries while being effectively separable at their own deviation levels.
• Figure 4: Ablation on Individual Distances.Left: CIFAR-10 Benchmark performance improves with an increasing number of distances. Right: ImageNet Benchmark performance improves with an increasing number of distances. The performance supports the use of all distances in our hyperparameter-free fDBD.
• Figure 5: Feature Distances to Decision Boundaries on a ResNet-18 CIFAR-10 Classifier. ID features tend to be further away from the decision boundaries compared to OOD features.

Theorems & Definitions (9)

• Definition 3.1
• Theorem 3.2
• proof
• Proposition 5.1
• Proposition 5.2
• proof
• proof
• Proposition 2.1
• proof