Comprehensive OOD Detection Improvements

TL;DR

The paper tackles out-of-distribution detection by jointly advancing representation-based and logit-based methods. It demonstrates that applying dimensionality reduction to penultimate representations can significantly boost OOD detection performance, and it introduces DICE-COL, a per-column masking variant of Directed Sparsification, to fix a flaw in the original DICE method. On OpenOODv1.5 benchmarks, the proposed approaches achieve state-of-the-art AUROC across multiple ID/OOD splits, with notable gains on CIFAR-10. The findings suggest that feature-space transformations and refined sparsification strategies can meaningfully enhance the reliability and efficiency of OOD detection in practical deployment.

Abstract

As machine learning becomes increasingly prevalent in impactful decisions, recognizing when inference data is outside the model's expected input distribution is paramount for giving context to predictions. Out-of-distribution (OOD) detection methods have been created for this task. Such methods can be split into representation-based or logit-based methods from whether they respectively utilize the model's embeddings or predictions for OOD detection. In contrast to most papers which solely focus on one such group, we address both. We employ dimensionality reduction on feature embeddings in representation-based methods for both time speedups and improved performance. Additionally, we propose DICE-COL, a modification of the popular logit-based method Directed Sparsification (DICE) that resolves an unnoticed flaw. We demonstrate the effectiveness of our methods on the OpenOODv1.5 benchmark framework, where they significantly improve performance and set state-of-the-art results.