Mitigating Overconfidence in Out-of-Distribution Detection by Capturing Extreme Activations

TL;DR

The paper tackles the problem of overconfidence in out-of-distribution (OOD) detection by introducing Capturing Extreme Activations (CEA), a proxy based on the $\\ell_2$-norm of penultimate-layer activations exceeding a threshold $\\tau$. By adding the CEA term with a trade-off parameter $\\lambda$ to the baseline novelty score, i.e., $f(x) + \\\lambda g(x)$, the method augments existing post-hoc OOD detectors without retraining. The authors validate CEA across diverse tabular and image datasets, architectures (ResNet, Transformer), and training losses (including LogitNorm), showing substantial AUC improvements on synthetic and real-world OODs while preserving in-distribution performance. Key insights include architecture- and data-heterogeneity-dependent effects, the compatibility of CEA with other overconfidence mitigation techniques, and the potential for broader applicability to time-series and text data.

Abstract

Detecting out-of-distribution (OOD) instances is crucial for the reliable deployment of machine learning models in real-world scenarios. OOD inputs are commonly expected to cause a more uncertain prediction in the primary task; however, there are OOD cases for which the model returns a highly confident prediction. This phenomenon, denoted as "overconfidence", presents a challenge to OOD detection. Specifically, theoretical evidence indicates that overconfidence is an intrinsic property of certain neural network architectures, leading to poor OOD detection. In this work, we address this issue by measuring extreme activation values in the penultimate layer of neural networks and then leverage this proxy of overconfidence to improve on several OOD detection baselines. We test our method on a wide array of experiments spanning synthetic data and real-world data, tabular and image datasets, multiple architectures such as ResNet and Transformer, different training loss functions, and include the scenarios examined in previous theoretical work. Compared to the baselines, our method often grants substantial improvements, with double-digit increases in OOD detection AUC, and it does not damage performance in any scenario.

Figures (6)

• Figure 1: Visual representation of the proposed method. We measure the $\ell_2$-norm of extreme activation values larger than the threshold $\tau$ (CEA) as an indicator of overconfidence caused by OOD samples and add it to the original novelty score computed based on the probabilities and activation values to generate the final novelty scores.
• Figure 2: OOD detection performance with and without CEA using the eICU (top) or Diabetics (bottom) datasets as ID and synthesized OOD data obtained by scaling. The blue bars are positioned in front of the red ones and cross markers are employed to emphasize the top of the red bars. The scaling factors $\alpha$ and baselines are presented under each bar.
• Figure 3: OOD detection performance with and without CEA using MIMIC-IV as ID and eICU as OOD (left) and the other way around (right). The blue bars are positioned in front of the red ones and cross markers are employed to emphasize the top of the red bars.
• Figure 4: Impact of parameters on the performance of CEA applied on different baseline OOD detection methods within the Diabetics dataset. (a) $\gamma=10$ and $p$ is changed. (b) $p=99.9$ and $\gamma$ is changed. The dashed lines indicate the performance of OOD detection methods without CEA ($\gamma=0$).
• Figure 5: OOD detection performance with and without CEA using the MIMIC-IV (top), Dry Bean (middle), and Wine Quality (bottom) datasets as ID and synthesized data by scaling. The blue bars are positioned in front of the red ones and cross markers are employed to emphasize the top of the red bars. The scaling factors and baseline names are presented under each bar.

Theorems & Definitions (6)

• Theorem 1
• proof
• Lemma 1
• proof
• Theorem 1
• proof