Outlier detection by ensembling uncertainty with negative objectness

TL;DR

This work reconsider direct prediction of K+1 logits that correspond to K groundtruth classes and one outlier class to formulate a novel anomaly score as an ensemble of in-distribution uncertainty and the posterior of the outlier class which is term negative objectness.

Abstract

Outlier detection is an essential capability in safety-critical applications of supervised visual recognition. Most of the existing methods deliver best results by encouraging standard closed-set models to produce low-confidence predictions in negative training data. However, that approach conflates prediction uncertainty with recognition of the negative class. We therefore reconsider direct prediction of K+1 logits that correspond to K groundtruth classes and one outlier class. This setup allows us to formulate a novel anomaly score as an ensemble of in-distribution uncertainty and the posterior of the outlier class which we term negative objectness. Now outliers can be independently detected due to i) high prediction uncertainty or ii) similarity with negative data. We embed our method into a dense prediction architecture with mask-level recognition over K+2 classes. The training procedure encourages the novel K+2-th class to learn negative objectness at pasted negative instances. Our models outperform the current state-of-the art on standard benchmarks for image-wide and pixel-level outlier detection with and without training on real negative data.

Figures (7)

• Figure 1: We propose UNO, a plug-in module that enables outlier-aware inference atop a desired feature extractor. UNO decouples in-distribution uncertainty from outlier recognition, and boosts the outlier detection performance by ensembling the two components.
• Figure 2: Interpretation of UNO in the pre-logit space $\mathbb{R}^d$ in an OpenOOD zhang2023openood experiment with CIFAR-10 inliers. Small $L_2$ norm of feature representations $\mathbf{z}$ leads to high $\textbf{s}_\text{Unc}$ while the angle between $\mathbf{z}$ and the K+1-th weight vector $\mathbf{w}_\text{K+1}$ leads to high $\textbf{s}_\text{NO}$. The two components capture different outliers as shown on the rightmost plot.
• Figure 3: Left: Visualization of real and synthetic negatives from an OpenOOD CIFAR-10 experiment zhang2023openood. Right: t-SNE plots of the corresponding feature representations. Our two-set training strategy yields synthetic negatives near the inlier manifold (a), while the naive approach collapses synthetic negatives to a single mode (b). Relative location of real negatives indicate that they cover similar modes of test outliers as our synthetic samples (c).
• Figure 4: Fine-tuning of dense classifier equipped with UNO. We paste negative training data (either real or synthetic) atop regular inlier images. The resulting mixed-content image is fed to the dense classifier that optimizes cross-entropy loss over K+2 classes.
• Figure 5: Qualitative experiments on Fishyscapes L&F val. Top row shows the input image and the three outlier scores. Bottom row shows anomaly detection maps after thresholding at TPR=95%. UNO significantly reduces the incidence of false-positive responses.