Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Revisiting Logit Distributions for Reliable Out-of-Distribution Detection

Jiachen Liang, Ruibing Hou, Minyang Hu, Hong Chang, Shiguang Shan, Xilin Chen

TL;DR

This work tackles reliable out-of-distribution detection in open-world settings by revisiting the information contained in the model's logits. It introduces LogitGap, a simple yet effective post-hoc scoring function that explicitly measures the gap between the maximum logit and the rest, preserving absolute logit magnitudes unlike softmax-based approaches. To combat noise from tail logits, the authors propose a focused top-N variant and an adaptive N-selection strategy that relies on a small ID validation set and synthetic OOD data generated via interpolation, formalizing the criterion as $ ext{arg max}_N ig(E_{ ext{OOD}}[ar{z}'_N] - E_{ ext{ID}}[ar{z}'_N]ig)$. Empirically, LogitGap and its adaptive version achieve state-of-the-art results across zero-shot, few-shot, and traditional OOD benchmarks on both vision-language (CLIP) and vision-only models, with notable improvements in challenging semantically hard and covariate-shift OOD tasks. The approach is lightweight, training-free, and readily integrable with pre-trained models, offering broad practical impact for reliable AI in open-world applications.

Abstract

Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning models in open-world applications. While post-hoc methods are favored for their efficiency and ease of deployment, existing approaches often underexploit the rich information embedded in the model's logits space. In this paper, we propose LogitGap, a novel post-hoc OOD detection method that explicitly exploits the relationship between the maximum logit and the remaining logits to enhance the separability between in-distribution (ID) and OOD samples. To further improve its effectiveness, we refine LogitGap by focusing on a more compact and informative subset of the logit space. Specifically, we introduce a training-free strategy that automatically identifies the most informative logits for scoring. We provide both theoretical analysis and empirical evidence to validate the effectiveness of our approach. Extensive experiments on both vision-language and vision-only models demonstrate that LogitGap consistently achieves state-of-the-art performance across diverse OOD detection scenarios and benchmarks. Code is available at https://github.com/GIT-LJc/LogitGap.

Revisiting Logit Distributions for Reliable Out-of-Distribution Detection

TL;DR

This work tackles reliable out-of-distribution detection in open-world settings by revisiting the information contained in the model's logits. It introduces LogitGap, a simple yet effective post-hoc scoring function that explicitly measures the gap between the maximum logit and the rest, preserving absolute logit magnitudes unlike softmax-based approaches. To combat noise from tail logits, the authors propose a focused top-N variant and an adaptive N-selection strategy that relies on a small ID validation set and synthetic OOD data generated via interpolation, formalizing the criterion as . Empirically, LogitGap and its adaptive version achieve state-of-the-art results across zero-shot, few-shot, and traditional OOD benchmarks on both vision-language (CLIP) and vision-only models, with notable improvements in challenging semantically hard and covariate-shift OOD tasks. The approach is lightweight, training-free, and readily integrable with pre-trained models, offering broad practical impact for reliable AI in open-world applications.

Abstract

Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning models in open-world applications. While post-hoc methods are favored for their efficiency and ease of deployment, existing approaches often underexploit the rich information embedded in the model's logits space. In this paper, we propose LogitGap, a novel post-hoc OOD detection method that explicitly exploits the relationship between the maximum logit and the remaining logits to enhance the separability between in-distribution (ID) and OOD samples. To further improve its effectiveness, we refine LogitGap by focusing on a more compact and informative subset of the logit space. Specifically, we introduce a training-free strategy that automatically identifies the most informative logits for scoring. We provide both theoretical analysis and empirical evidence to validate the effectiveness of our approach. Extensive experiments on both vision-language and vision-only models demonstrate that LogitGap consistently achieves state-of-the-art performance across diverse OOD detection scenarios and benchmarks. Code is available at https://github.com/GIT-LJc/LogitGap.
Paper Structure (46 sections, 3 theorems, 35 equations, 5 figures, 17 tables)

This paper contains 46 sections, 3 theorems, 35 equations, 5 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary
  4. Problem Setting
  5. OOD Detection Scoring Function
  6. Method
  7. OOD score: LogitGap
  8. Logits Selection for Focused Scoring
  9. Experiments
  10. Experimental Settings
  11. Results
  12. Further Discussion
  13. Influence of the Hyperparameter $N$ on LogitGap Performance.
  14. Logit Distribution Differences Between ID and OOD Data.
  15. Different Variants of LogitGap.
  16. ...and 31 more sections

Key Result

Theorem 4.1

Given a $K$-way classification task, let the predicted logit vector for a sample $\boldsymbol{x}$ be $\boldsymbol{z} = \left[z_1, z_2, \ldots, z_K\right]$. Let $\boldsymbol{z}' = \left[z'_1, z'_2, \ldots, z'_K\right]$ denote the sorted logits in descending order, such that $z'_1 = \max_{k} z_k$. The where $\operatorname{FPR_{LogitGap}}\left(\lambda_\text{L}\right)$ is the false positive rate based

Figures (5)

  • Figure 1: OOD score distributions for ID and OOD samples across scoring functions. The x-axis shows predicted class indices. Dots represent mean scores per class; vertical lines indicate score ranges. Gray shading marks overlap regions of ID and OOD scores. Results are reported using CLIP ViT-B/16 model on ImageNet-20 (ID) and ImageNet-10 (OOD).
  • Figure 2: Descending-sorted logits on CLIP ViT-B/16 with ImageNet100 (ID) and iNaturalist (OOD).
  • Figure 3: Variation of FPR95 performance with increasing $N$. Results are reported on CLIP ViT-B/16 with ImageNet-100 as the ID dataset in semantic shift senario.
  • Figure 4: Logit Statistics on ViT-L/14 with ImageNet as ID. $z'_1$ and $\bar{z}'_N$ denote the average maximum logit and the average logit of non-predicted classes, respectively.
  • Figure 5: Descending-sorted logits on CLIP ViT-B/16 with ImageNet100 (ID) and iNaturalist (OOD).

Theorems & Definitions (6)

  • Theorem 4.1
  • proof : proof sketch
  • Theorem A.1
  • proof
  • Theorem B.1
  • proof