Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detecting OOD Samples via Optimal Transport Scoring Function

Heng Gao, Zhuolin He, Jian Pu

TL;DR

This work tackles the challenge of detecting out-of-distribution samples without relying on additional training data. It introduces OTOD, a post hoc scoring function based on Wasserstein-1 distance that aggregates information from penultimate features, logits, and softmax probabilities, with temperature scaling to amplify in-distribution/out-of-distribution gaps. The method provides theoretical guarantees for the feature component and demonstrates superior empirical performance on CIFAR-10/100 benchmarks against multiple baselines. The approach is simple to deploy (no retraining) and leverages optimal transport to capture geometric shifts in representation space, offering practical benefits for real-world OOD detection systems.

Abstract

To deploy machine learning models in the real world, researchers have proposed many OOD detection algorithms to help models identify unknown samples during the inference phase and prevent them from making untrustworthy predictions. Unlike methods that rely on extra data for outlier exposure training, post hoc methods detect Out-of-Distribution (OOD) samples by developing scoring functions, which are model agnostic and do not require additional training. However, previous post hoc methods may fail to capture the geometric cues embedded in network representations. Thus, in this study, we propose a novel score function based on the optimal transport theory, named OTOD, for OOD detection. We utilize information from features, logits, and the softmax probability space to calculate the OOD score for each test sample. Our experiments show that combining this information can boost the performance of OTOD with a certain margin. Experiments on the CIFAR-10 and CIFAR-100 benchmarks demonstrate the superior performance of our method. Notably, OTOD outperforms the state-of-the-art method GEN by 7.19% in the mean FPR@95 on the CIFAR-10 benchmark using ResNet-18 as the backbone, and by 12.51% in the mean FPR@95 using WideResNet-28 as the backbone. In addition, we provide theoretical guarantees for OTOD. The code is available in https://github.com/HengGao12/OTOD.

Detecting OOD Samples via Optimal Transport Scoring Function

TL;DR

This work tackles the challenge of detecting out-of-distribution samples without relying on additional training data. It introduces OTOD, a post hoc scoring function based on Wasserstein-1 distance that aggregates information from penultimate features, logits, and softmax probabilities, with temperature scaling to amplify in-distribution/out-of-distribution gaps. The method provides theoretical guarantees for the feature component and demonstrates superior empirical performance on CIFAR-10/100 benchmarks against multiple baselines. The approach is simple to deploy (no retraining) and leverages optimal transport to capture geometric shifts in representation space, offering practical benefits for real-world OOD detection systems.

Abstract

To deploy machine learning models in the real world, researchers have proposed many OOD detection algorithms to help models identify unknown samples during the inference phase and prevent them from making untrustworthy predictions. Unlike methods that rely on extra data for outlier exposure training, post hoc methods detect Out-of-Distribution (OOD) samples by developing scoring functions, which are model agnostic and do not require additional training. However, previous post hoc methods may fail to capture the geometric cues embedded in network representations. Thus, in this study, we propose a novel score function based on the optimal transport theory, named OTOD, for OOD detection. We utilize information from features, logits, and the softmax probability space to calculate the OOD score for each test sample. Our experiments show that combining this information can boost the performance of OTOD with a certain margin. Experiments on the CIFAR-10 and CIFAR-100 benchmarks demonstrate the superior performance of our method. Notably, OTOD outperforms the state-of-the-art method GEN by 7.19% in the mean FPR@95 on the CIFAR-10 benchmark using ResNet-18 as the backbone, and by 12.51% in the mean FPR@95 using WideResNet-28 as the backbone. In addition, we provide theoretical guarantees for OTOD. The code is available in https://github.com/HengGao12/OTOD.

Paper Structure

This paper contains 13 sections, 1 theorem, 10 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Wasserstein Distances
  4. Optimal Transport Scoring Function
  5. Optimal Transport Score Calculation
  6. Multi-scale Information Integration
  7. Theoretical Guarantee
  8. Experimental Results
  9. Experimental Setup
  10. OOD Detection Performance Results
  11. Ablation Analysis of OTOD Input
  12. The Effect of Hyperparameters
  13. Conclusion

Key Result

Theorem 1

Suppose that the diameter of $\Tilde{X}$ is bounded by $D$, $V(\Tilde{X})$ is the volume of $\Tilde{X}$. Consider the case where $\Tilde{X}$ is the normalized feature space. Then, $\forall 1\leq i\leq K,$ given the setups above, we have the following inequality:

Figures (3)

  • Figure 1: The AUROC (in percentage) of four OOD detection methods using ResNet-18 b5 trained on CIFAR-10 b6. The OOD datasets are Tiny ImageNet (TIN) b11 and Texture b14. Methods marked with blue $\lozenge$ use the softmax probability input; methods marked with orange $\circ$ use the logits inputs. Our proposed method OTOD (marked with purple $\star$) uses information from logits, softmax probability, and features.
  • Figure 2: UMAP b22 visualization of unnormalized (a) and normalized (b) feature distribution using WideResNet-28 b16 on the CIFAR-10 b3 benchmark.
  • Figure 3: (a)(b) Hyperparameter analysis on temperature $T$ using ResNet-18 and WideResNet-28 as backbones on the CIFAR-100 benchmark.

Theorems & Definitions (4)

  • Definition 1: Wasserstein distances b7
  • Definition 2: Mean Discrepancy
  • Theorem 1: Mean Discrepancy Upper Bound
  • proof