Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Closer Look at the Learnability of Out-of-Distribution (OOD) Detection

Konstantin Garov, Kamalika Chaudhuri

TL;DR

This work reframes OOD detection as a PAC-style learning problem and distinguishes uniform from non-uniform learnability. By introducing structured assumptions such as Disjoint Supports Assumption (DSA), τ-Far Supports (τ-FSA), ConvexID, ContID, and Hölder continuity, it identifies conditions under which OOD detection is learnable and provides concrete algorithms with sample-complexity analyses when possible. A key contribution is the principled Maximal Zero OOD Risk Procedure and its variants, which yield zero OOD risk under several assumptions and illuminate when uniform learnability is achievable (e.g., with τ-FSA plus bounded-in-distribution diameter or Hölder-continuous distributions combined with DSA). The results also extend No Free Lunch-type theorems to OOD detection, showing impossibilities in broad settings, while explaining why practical detectors often succeed in real data due to favorable distributional properties. Overall, the paper clarifies the theoretical landscape of OOD detection, bridging theory and practice, and outlining directions for future work under weaker or alternative modeling assumptions.

Abstract

Machine learning algorithms often encounter different or "out-of-distribution" (OOD) data at deployment time, and OOD detection is frequently employed to detect these examples. While it works reasonably well in practice, existing theoretical results on OOD detection are highly pessimistic. In this work, we take a closer look at this problem, and make a distinction between uniform and non-uniform learnability, following PAC learning theory. We characterize under what conditions OOD detection is uniformly and non-uniformly learnable, and we show that in several cases, non-uniform learnability turns a number of negative results into positive. In all cases where OOD detection is learnable, we provide concrete learning algorithms and a sample-complexity analysis.

A Closer Look at the Learnability of Out-of-Distribution (OOD) Detection

TL;DR

This work reframes OOD detection as a PAC-style learning problem and distinguishes uniform from non-uniform learnability. By introducing structured assumptions such as Disjoint Supports Assumption (DSA), τ-Far Supports (τ-FSA), ConvexID, ContID, and Hölder continuity, it identifies conditions under which OOD detection is learnable and provides concrete algorithms with sample-complexity analyses when possible. A key contribution is the principled Maximal Zero OOD Risk Procedure and its variants, which yield zero OOD risk under several assumptions and illuminate when uniform learnability is achievable (e.g., with τ-FSA plus bounded-in-distribution diameter or Hölder-continuous distributions combined with DSA). The results also extend No Free Lunch-type theorems to OOD detection, showing impossibilities in broad settings, while explaining why practical detectors often succeed in real data due to favorable distributional properties. Overall, the paper clarifies the theoretical landscape of OOD detection, bridging theory and practice, and outlining directions for future work under weaker or alternative modeling assumptions.

Abstract

Machine learning algorithms often encounter different or "out-of-distribution" (OOD) data at deployment time, and OOD detection is frequently employed to detect these examples. While it works reasonably well in practice, existing theoretical results on OOD detection are highly pessimistic. In this work, we take a closer look at this problem, and make a distinction between uniform and non-uniform learnability, following PAC learning theory. We characterize under what conditions OOD detection is uniformly and non-uniformly learnable, and we show that in several cases, non-uniform learnability turns a number of negative results into positive. In all cases where OOD detection is learnable, we provide concrete learning algorithms and a sample-complexity analysis.
Paper Structure (32 sections, 33 theorems, 74 equations, 1 figure, 1 table)

This paper contains 32 sections, 33 theorems, 74 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Some Necessary Definitions
  4. A Summary of the Learnability Results
  5. Assumptions
  6. Separation Assumptions.
  7. Structural Assumptions.
  8. Distributional Assumptions.
  9. A Principled Learner
  10. Summary of the Results
  11. OOD detection and VC-dimension
  12. Results of Table 1
  13. The Disjoint Supports Assumption is not enough for OOD Detection
  14. Far-OOD Detection
  15. OOD Detection of Hölder Continuous Distributions
  16. ...and 17 more sections

Key Result

Proposition 1

Let $\mathcal{D}$ be a domain space over $\mathcal{X}$ and let $\mathcal{B}: \mathcal{X}^* \rightarrow \mathcal{P}(\mathcal{X})$ be a procedure such that for all domains $D \in \mathcal{D}$ the ID risk of $\mathcal{B}(S)$ converges in probability to $0$ as the number of samples tends to infinity. Ad holds for all domains $D$ in $\mathcal{D}$. Then OOD detection for $\mathcal{D}$ is non-uniformly l

Figures (1)

  • Figure 1: Examples of domain restrictions

Theorems & Definitions (64)

  • Definition 1: Uniform learnability of OOD detection
  • Definition 2: Non-uniform learnability of OOD detection
  • Definition 3
  • Proposition 1
  • Definition 4
  • Theorem 1: No Free Lunch Theorem for OOD Detection
  • Theorem 2
  • Theorem 3: Far-OOD Detection
  • Theorem 4
  • Theorem 5
  • ...and 54 more