EOOD: Entropy-based Out-of-distribution Detection

TL;DR

EOOD tackles OOD overconfidence by exploiting differences in information flow across network blocks. It computes blockwise conditional entropy f^{CE}(x,l)=H(B^{(l-1)}|B^{(l)}) and uses a CER-based selection, based on ID and pseudo-OOD samples, to pick the most sensitive block l^* and define the final score Score_OOD(x)=f^{CE}(x,l^*). The approach achieves state-of-the-art post-hoc OOD detection performance on standard CIFAR and ImageNet benchmarks without retraining or auxiliary OOD data, demonstrating robustness across small- and large-scale settings and highlighting the value of entropy-based, block-level information-flow signals for open-world reliability.

Abstract

Deep neural networks (DNNs) often exhibit overconfidence when encountering out-of-distribution (OOD) samples, posing significant challenges for deployment. Since DNNs are trained on in-distribution (ID) datasets, the information flow of ID samples through DNNs inevitably differs from that of OOD samples. In this paper, we propose an Entropy-based Out-Of-distribution Detection (EOOD) framework. EOOD first identifies specific block where the information flow differences between ID and OOD samples are more pronounced, using both ID and pseudo-OOD samples. It then calculates the conditional entropy on the selected block as the OOD confidence score. Comprehensive experiments conducted across various ID and OOD settings demonstrate the effectiveness of EOOD in OOD detection and its superiority over state-of-the-art methods.

Figures (4)

• Figure 1: Visualization of conditional entropy values on WideResNet-28 model. The CIFAR-10 dataset is used as ID, and the TinyImageNet-R dataset is used as OOD. Left: The conditional entropy distribution of the last Block. Right: The conditional entropy distribution of the $10$-th Block selected using the conditional entropy ratio. The $x$-axis represents the conditional entropy computed by $f^{CE}(\cdot, \cdot)$ from Eq. \ref{['eq5']}, and the $y$-axis represents the frequency of occurrence.
• Figure 2: Illustration of our proposed out-of-distribution detection framework. (a) Inference path in the neural network, where the input passes through blocks and fully connected (FC) layers to reach the final classification. (b) Select the block with the largest conditional entropy ratio. (c) Calculate the conditional entropy of the selected block as $\text{Score}_{OOD}$.
• Figure 3: Comparison of EOOD's OOD detection performance using different blocks in WideResNet-28 trained on CIFAR-10 and WideResNet-50 trained on ImageNet-1k, with FPR95 and AUROC metrics averaged over multiple OOD datasets.
• Figure 4: Computing the CER under different jigsaw puzzle configurations using WideResNet-28 model trained on CIFAR-10.