Feature Density Estimation for Out-of-Distribution Detection via Normalizing Flows

TL;DR

The paper tackles safe open-world deployment by improving out-of-distribution detection through feature-density estimation using normalizing flows on representations from frozen pretrained backbones. By training a lightweight flow for density estimation on normalized feature vectors for just one epoch, the method achieves strong far-OOD detection, surpassing prior approaches on large-scale benchmarks (e.g., ImageNet-1k Textures). A key finding is that the backbone's feature-space distribution, particularly its tolerance and uniformity, strongly influences OOD performance, with under-training and feature normalization yielding robust results. The work demonstrates that normalizing flows can be effective post-hoc tools for OOD detection, offering practical benefits for safety-critical systems in complex, open environments.

Abstract

Out-of-distribution (OOD) detection is a critical task for safe deployment of learning systems in the open world setting. In this work, we investigate the use of feature density estimation via normalizing flows for OOD detection and present a fully unsupervised approach which requires no exposure to OOD data, avoiding researcher bias in OOD sample selection. This is a post-hoc method which can be applied to any pretrained model, and involves training a lightweight auxiliary normalizing flow model to perform the out-of-distribution detection via density thresholding. Experiments on OOD detection in image classification show strong results for far-OOD data detection with only a single epoch of flow training, including 98.2% AUROC for ImageNet-1k vs. Textures, which exceeds the state of the art by 7.8%. We additionally explore the connection between the feature space distribution of the pretrained model and the performance of our method. Finally, we provide insights into training pitfalls that have plagued normalizing flows for use in OOD detection.

Figures (6)

• Figure 1: Feature density estimation for out-of-distribution detection.
• Figure 2: (a) A t-SNE visualization of the feature space of a ResNet18 backbone model comparing ID data (CIFAR-10), OOD data (SVHN), and flow generated feature vectors. Clearly visible are 10 clusters corresponding to the 10 classes of CIFAR-10, and the coincidence between the ID data and the flow generated data points. (b) A histogram of the log-likelihood of ID features vs. OOD features under the normalizing flow model. Better separability of these distributions leads to higher AUROC for OOD detection.
• Figure 3: Feature likelihood histograms for the same flow model at 0 epochs and 999 epochs. With further training, the likelihood increases for all distributions, but the training and validation distributions begin to separate due to overfitting, while the separability of the ID/OOD distributions degrades.
• Figure 4: Normalizing flow AUROC (CIFAR-10 vs. SVHN) and loss during training. Loss of ID and OOD test datasets decreases during training. AUROC peaks early then declines.
• Figure 5: Visualization of feature vector norms vs. log-likelihood for a flow model trained with normalized (left) and unnormalized (right) feature vectors. For a flow model trained on unnormalized features, there is no correlation between feature norm, classification accuracy, and flow likelihood. For a flow model trained on normalized features, a correlation is observed between feature norm, classification accuracy, and flow likelihood.