ProHOC: Probabilistic Hierarchical Out-of-Distribution Classification via Multi-Depth Networks

TL;DR

ProHOC tackles hierarchical out-of-distribution detection by placing OOD samples at internal nodes of a given class hierarchy and ID samples at leaves. It builds a probabilistic model on an extended hierarchy G and uses multi-depth networks to approximate the hierarchy conditionals p(y|c,x) at each depth, yielding a predictive distribution p(c|x) over both ID leaves and OOD nodes. By evaluating p(c|x) with an objective based on the expected hierarchical distance dist_H, ProHOC makes informative predictions, with two scoring variants—CompProb and EntCompProb (and their combination EntCompProb)—to model OOD probabilities. Experiments on FGVC-Aircraft, iNaturalist19, and SimpleHierImageNet show ProHOC outperforms prior hierarchical OOD methods on MixBMHD and MixBAcc, and demonstrates compatibility with stronger backbones such as DINOv2 ViT, all without introducing extra hyperparameters. This framework provides a principled, scalable approach for semantically aware OOD classification in hierarchical label spaces, with potential impact on safety-critical deployment of vision systems.

Abstract

Out-of-distribution (OOD) detection in deep learning has traditionally been framed as a binary task, where samples are either classified as belonging to the known classes or marked as OOD, with little attention given to the semantic relationships between OOD samples and the in-distribution (ID) classes. We propose a framework for detecting and classifying OOD samples in a given class hierarchy. Specifically, we aim to predict OOD data to their correct internal nodes of the class hierarchy, whereas the known ID classes should be predicted as their corresponding leaf nodes. Our approach leverages the class hierarchy to create a probabilistic model and we implement this model by using networks trained for ID classification at multiple hierarchy depths. We conduct experiments on three datasets with predefined class hierarchies and show the effectiveness of our method. Our code is available at https://github.com/walline/prohoc.

Figures (14)

• Figure 1: Out-of-distribution detection in class hierarchies. Instead of simply predicting the unseen dog and cat types as OOD, we aim to classify them as the high-level categories dog and cat.
• Figure 2: For a hierarchy of depth 3, we train separate neural networks, parameterized by $\theta_1$, $\theta_2$, and $\theta_3$, to classify data into nodes within that depth level.
• Figure 3: Prediction example: $\text{dist}_\mathcal{H}(\text{LCA}, f(x)) = 2$, $\text{dist}_\mathcal{H}(\text{LCA}, y) = 1$.
• Figure 4: Prediction example: $\text{dist}_\mathcal{H}(\text{LCA}, f(x)) = 2$, $\text{dist}_\mathcal{H}(\text{LCA}, y) = 0$. This represents a pure overprediction.
• Figure 5: Prediction example: $\text{dist}_\mathcal{H}(\text{LCA}, f(x)) = 0$, $\text{dist}_\mathcal{H}(\text{LCA}, y) = 1$. This represents a pure underprediction.