Multi-label out-of-distribution detection via evidential learning

TL;DR

The paper tackles multi-label out-of-distribution detection in visual recognition by introducing a Beta Evidential Neural Network (Beta-ENN) that jointly models likelihood and predictive uncertainty via a Beta distribution over label probabilities, enabling two uncertainty-based OOD scoring schemes. It defines OOD Score - Max and OOD Score - Sum, leveraging positive and negative evidence to detect novel data across multiple labels, with the mean per-label probability given by $p_l = α_l/(α_l+β_l)$. The method is validated on PASCAL-VOC, MS-COCO, and NUS-WIDE, outperforming state-of-the-art posthoc detectors, particularly when using the Sum criterion and the combined $U_{p,n}^s$ score. The results suggest that incorporating both positive and negative evidences improves robustness to OOD in multi-label scenarios, with potential for extension to continual learning settings. The work advances practical, uncertainty-aware multi-label OOD detection for real-world visual recognition systems.

Abstract

A crucial requirement for machine learning algorithms is not only to perform well, but also to show robustness and adaptability when encountering novel scenarios. One way to achieve these characteristics is to endow the deep learning models with the ability to detect out-of-distribution (OOD) data, i.e. data that belong to distributions different from the one used during their training. It is even a more complicated situation, when these data usually are multi-label. In this paper, we propose an approach based on evidential deep learning in order to meet these challenges applied to visual recognition problems. More concretely, we designed a CNN architecture that uses a Beta Evidential Neural Network to compute both the likelihood and the predictive uncertainty of the samples. Based on these results, we propose afterwards two new uncertainty-based scores for OOD data detection: (i) OOD - score Max, based on the maximum evidence; and (ii) OOD score - Sum, which considers the evidence from all outputs. Extensive experiments have been carried out to validate the proposed approach using three widely-used datasets: PASCAL-VOC, MS-COCO and NUS-WIDE, demonstrating its outperformance over several State-of-the-Art methods.

Figures (6)

• Figure 1: Overview of the Beta Evidential Neural Network proposed for multi-label recognition and OOD detection.
• Figure 2: Total of in- and out- of distribution instances used to evaluate the performance on each dataset.
• Figure 3: ROC curve for OOD data detection. The first, second, and third rows correspond to the OOD performance on PASCAL-VOC, MS-COCO and NUS-WIDE, respectively. For each OOD score, the results are presented when using the Max (left column) or Sum (right column) criteria.
• Figure 4: OOD detection performance by varying the value of $\lambda_2$ in the proposed $U_{p,n}^s$ approach. From left to right: PASCAL-VOC, MS-COCO and NUS-WIDE.
• Figure 5: Examples of OOD detection failures using JointEnergy on MS-COCO/ImageNet-1K datasets. Barplots represent the logit output.