Out-of-Distribution Detection & Applications With Ablated Learned Temperature Energy

TL;DR

The paper introduces Ablated Learned Temperature Energy (AbeT), an OOD detection method that replaces a single scalar temperature with an input-dependent learned temperature and removes the Forefront Temperature Constant from the energy score. AbeT achieves state-of-the-art or competitive results across classification, semantic segmentation, and object detection, notably reducing FPR@95 and increasing AUROC/AUPRC on standard OOD benchmarks. The authors provide empirical and visual evidence that leveraging misclassified ID examples during training helps AbeT distinguish ID from OOD without using explicit OOD data. The approach is lightweight, requiring only a small architectural change (a learned temperature module) and a targeted ablation, with potential for extension to large-language models and vision-language models.

Abstract

As deep neural networks become adopted in high-stakes domains, it is crucial to identify when inference inputs are Out-of-Distribution (OOD) so that users can be alerted of likely drops in performance and calibration despite high confidence -- ultimately to know when networks' decisions (and their uncertainty in those decisions) should be trusted. In this paper we introduce Ablated Learned Temperature Energy (or "AbeT" for short), an OOD detection method which lowers the False Positive Rate at 95\% True Positive Rate (FPR@95) by $43.43\%$ in classification compared to state of the art without training networks in multiple stages or requiring hyperparameters or test-time backward passes. We additionally provide empirical insights as to why our model learns to distinguish between In-Distribution (ID) and OOD samples while only being explicitly trained on ID samples via exposure to misclassified ID examples at training time. Lastly, we show the efficacy of our method in identifying predicted bounding boxes and pixels corresponding to OOD objects in object detection and semantic segmentation, respectively -- with an AUROC increase of $5.15\%$ in object detection and both a decrease in FPR@95 of $41.48\%$ and an increase in AUPRC of $34.20\%$ in semantic segmentation compared to previous state of the art.

Figures (5)

• Figure 1: Histograms showing the separability between OOD scores on OOD inputs (red) and ID inputs (blue) for different methods. The goal is to make these red and blue distributions as separable as possible, with scores on OOD inputs (red) close to 0 and scores on ID inputs (blue) of high-magnitude (away from 0). (Center) Our first contribution is replacing the Scalar Temperature in the Energy Score liu2020energy with a Learned Temperature hsu2020generalized. This infusion leads to Equation \ref{['eqn:two_temperature']}, with the Learned Temperature showing up in the Exponential Divisor Temperature (overlined in Equation \ref{['eqn:two_temperature']}) and Forefront Temperature Constant (underlined in Equation \ref{['eqn:two_temperature']}) forms. (Right) The Forefront Temperature Constant contradicts the desired property of scores being close to 0 for OOD points (red) and of high magnitude for ID points (blue). (Left) Therefore, our second contribution is to ablate this Forefront Temperature Constant, leading to our final Ablated Learned Temperature Energy ( AbeT) score. This ablation increases the separability of the OOD scores vs. ID scores, as can be seen visually and numerically (in terms of AUROC) comparing the center and left plots - where the only difference is this ablation of the Forefront Temperature Constant. Higher AUROC means more separability.
• Figure 2: Qualitative comparison of OOD scores for semantic segmentation. The top row and bottom row contain examples from the datasets RoadAnomaly lis2019roadanomaly and LostAndFound pinggera2016laf, respectively. Pixels corresponding to OOD objects are highlighted in red in each image in the leftmost column, which are cropped to regions where we have ID/OOD labels. Scores for each example (row) and technique (column) are thresholded at their respective 95% True Positive Rate and then normalized $[0,1]$ in the red channel, with void pixels (which have no ID/OOD label) set to 0. Bright red pixels represent high OOD scores, which should cover the same region as the pixels which correspond to OOD objects in the leftmost column. We invert the scores of Standardized Max Logit, Max Logit, and MSP to allow these methods to highlight OOD pixels in red.
• Figure 3: The network architecture of a classification network with a learned temperature.
• Figure 4: Performance of AbeT with input perturbation. This shows our method using input perturbations from ODIN liang2017enhancing. The x-axis is different perturbation magnitudes, and the y-axis is FPR@95 (lower is better). A ResNetv2-101 was trained on ImageNet-1k Krizhevsky09learningmultiple. For three of the OOD datasets, adding any perturbation hurts performance. For Places365, adding in low levels of perturbation slightly improves performance.
• Figure 5: (Left) Scatter plot of OOD LSUN examples (red) and ID test CIFAR-10 examples (blue). (Center) ID test CIFAR-10 examples correctly classified (blue) and incorrectly classified (red). (Right) ID test CIFAR-10 examples colored by their AbeT score. Red is estimated to be more OOD. The learned temperature increasing on misclassified points (in order to deflate softmax confidence when incorrect) leads our score to inflate towards 0 on misclassified points, as can be seen in the center plot. The presence of a comparatively higher proportion of points in the center of penultimate representation space which are misclassified therefore leads to the relationship that our our score inflates towards 0 as distance to the center decreases (as can be seen on ID points in the right plot). In combination with OOD points lying in the center of penultimate representation space (as can be seen on the left plot), this means that our scores are close to 0 on OOD points - thus providing intuition (but not proof) as to why our method is able to achieve superior OOD detection performance.