Out-of-Distribution Detection with Prototypical Outlier Proxy

TL;DR

This paper tackles the problem of deep models producing over-confident predictions on out-of-distribution data. It introduces Prototypical Outlier Proxy (POP), a fixed-classifier framework augmented with virtual outlier proxies and a hierarchy-aware loss (HSBL) to reshape decision boundaries without exposing the model to real or synthesized outliers. By deriving ID prototypes from a semantic hierarchy and placing outlier proxies in a structured way, POP achieves strong OOD detection performance across CIFAR-10, CIFAR-100, and ImageNet-200, while also offering substantial speed advantages over prior outlier synthesis methods like NPOS. The approach demonstrates robustness across near- and far-OOD scenarios and maintains scalability to large datasets, reinforcing its practical applicability for real-world deployments.

Abstract

Out-of-distribution (OOD) detection is a crucial task for deploying deep learning models in the wild. One of the major challenges is that well-trained deep models tend to perform over-confidence on unseen test data. Recent research attempts to leverage real or synthetic outliers to mitigate the issue, which may significantly increase computational costs and be biased toward specific outlier characteristics. In this paper, we propose a simple yet effective framework, Prototypical Outlier Proxy (POP), which introduces virtual OOD prototypes to reshape the decision boundaries between ID and OOD data. Specifically, we transform the learnable classifier into a fixed one and augment it with a set of prototypical weight vectors. Then, we introduce a hierarchical similarity boundary loss to impose adaptive penalties depending on the degree of misclassification. Extensive experiments across various benchmarks demonstrate the effectiveness of POP. Notably, POP achieves average FPR95 reductions of 7.70%, 6.30%, and 5.42% over the second-best methods on CIFAR-10, CIFAR-100, and ImageNet-200, respectively. Moreover, compared to the recent method NPOS, which relies on outlier synthesis, POP trains 7.2X faster and performs inference 19.5X faster. The source code is available at: https://github.com/gmr523/pop.

Figures (7)

• Figure 1: Illustration of our motivation. Left: Vanilla training. Middle: Training with the mixture of ID data and outliers. Right: Training with prototypical outlier proxies.
• Figure 2: FPR95 (%) of six OOD detection baselines and our POP, using ResNet-18 trained on CIFAR-10, tested on CIFAR-100 and MNIST. Lower FPR95 values indicate better performance. Blue $\triangle$ denotes real outliers, orange $\square$ denotes synthetic outliers, and red $\star$ is our POP. 'near' and 'far' indicate the degree of difference between ID and OOD data.
• Figure 3: Toy example. Use a ResNet-18 with a feature layer size of 2 for three CIFAR-10 classes. The $x$- and $y$-axes represent the feature values in the square region. We evaluate prediction confidence for each point in these classes. Yellow, green, and blue points represent deer, horse, and ship, respectively. (a) Vanilla ResNet-18. (b) Fixed ResNet-18. (c) Fixed ResNet-18 with one outlier proxy (brown star marker).
• Figure 4: The overview of POP. The contributions module in POP is colored. Before training, in the green module at the bottom (①), integrate prototypical outlier proxies into the fixed classifier. Then, ID data is fed into the model for learning (②). Finally, during the test phase (③), OOD data is fed into the model, and the OOD score is calculated using the feature norm and logits.
• Figure 5: The principles of HSBL