VOS: Learning What You Don't Know by Virtual Outlier Synthesis

TL;DR

This work tackles the lack of supervision for unknown data in OOD detection by introducing VOS, a framework that synthesizes virtual outliers directly in feature space to regularize the ID-OOD boundary. It combines an unknown-aware energy-based training objective with per-class Gaussian modeling and online feature-space sampling, extended to object-level detection. The approach yields state-of-the-art or competitive OOD detection performance on both object detection and image classification benchmarks while preserving in-distribution accuracy, and it operates without relying on real outlier datasets. The authors provide extensive ablations, show practical benefits for safety-critical applications, and release code to support reproducibility.

Abstract

Out-of-distribution (OOD) detection has received much attention lately due to its importance in the safe deployment of neural networks. One of the key challenges is that models lack supervision signals from unknown data, and as a result, can produce overconfident predictions on OOD data. Previous approaches rely on real outlier datasets for model regularization, which can be costly and sometimes infeasible to obtain in practice. In this paper, we present VOS, a novel framework for OOD detection by adaptively synthesizing virtual outliers that can meaningfully regularize the model's decision boundary during training. Specifically, VOS samples virtual outliers from the low-likelihood region of the class-conditional distribution estimated in the feature space. Alongside, we introduce a novel unknown-aware training objective, which contrastively shapes the uncertainty space between the ID data and synthesized outlier data. VOS achieves competitive performance on both object detection and image classification models, reducing the FPR95 by up to 9.36% compared to the previous best method on object detectors. Code is available at https://github.com/deeplearning-wisc/vos.

Figures (10)

• Figure 1: (a) A Faster-RCNN ren2015faster model trained on BDD-100k dataset DBLP:conf/cvpr/YuCWXCLMD20 produces overconfident predictions for OOD object (e.g., moose). (b)-(c) The uncertainty measurement with and without virtual outlier training. The in-distribution data $\mathbf{x}\in \altmathcal{X}=\mathbb{R}^2$ is sampled from a Gaussian mixture model). Regularizing the model with virtual outliers (c) better captures the OOD uncertainty than without (b).
• Figure 2: The framework of VOS. We model the feature representation of ID objects as class-conditional Gaussians, and sample virtual outliers $\mathbf{v}$ from the low-likelihood region. The virtual outliers, along with the ID objects, are used to produce the uncertainty loss for regularization. The uncertainty estimation branch ($\altmathcal{L}_{\mathrm{uncertainty}}$) is jointly trained with the object detection loss ($\altmathcal{L}_{\mathrm{loc}},\altmathcal{L}_{\mathrm{cls}}$).
• Figure 3: UMAP visualization of feature embeddings of PASCAL-VOC (on a subset of 10 classes).
• Figure 4: Visualization of detected objects on the OOD images (from MS-COCO) by a vanilla Faster-RCNN (top) and VOS (bottom). The in-distribution is BDD-100k dataset. Blue: Objects detected and classified as one of the ID classes. Green: OOD objects detected by VOS, which reduce false positives among detected objects.
• Figure 5: Additional visualization of detected objects on the OOD images (from MS-COCO) by a vanilla Faster-RCNN (top) and VOS (bottom). The in-distribution is Pascal VOC dataset. Blue: Objects detected and classified as one of the ID classes. Green: OOD objects detected by VOS, which reduce false positives among detected objects.