Energy Correction Model in the Feature Space for Out-of-Distribution Detection

TL;DR

Problem: OOD detection in the feature space of a pre-trained classifier. Approach: a Gaussian-mixture reference density q(z) = ∑_c π_c N(z; μ_c, Σ) is refined by an energy-based correction p_theta(z) ∝ exp(- (E_theta(z) + E_Maha(z))). Contributions: first demonstration that training an EBM in feature space is competitive, introduces the energy-based correction to refine MoG, and shows favorable CIFAR-10/100 results against baselines. Findings: improves near and mid-OOD detection and mitigates MCMC non-mixing issues, offering a practical density-estimation approach for OOD detection in real classifiers. Significance: provides a scalable method to perform density estimation in feature space for robust OOD detection across datasets and backbones.

Abstract

In this work, we study the out-of-distribution (OOD) detection problem through the use of the feature space of a pre-trained deep classifier. We show that learning the density of in-distribution (ID) features with an energy-based models (EBM) leads to competitive detection results. However, we found that the non-mixing of MCMC sampling during the EBM's training undermines its detection performance. To overcome this an energy-based correction of a mixture of class-conditional Gaussian distributions. We obtains favorable results when compared to a strong baseline like the KNN detector on the CIFAR-10/CIFAR-100 OOD detection benchmarks.

Figures (8)

• Figure 1: Toy dataset
• Figure 2: Gaussian
• Figure 3: Energy correction
• Figure 4: ours
• Figure 6: Dataset