Evidential Deep Learning to Quantify Classification Uncertainty

TL;DR

This work addresses the lack of calibrated prediction uncertainty in deterministic classifiers by modeling class probabilities with a Dirichlet posterior learned via a standard neural net. The network outputs nonnegative evidence f(x;Theta), forming Dirichlet parameters alpha = f(x;Theta) + 1, with predictive uncertainty captured through the Dirichlet distribution D(p|alpha). A specially designed loss combines a mean-squared-error term on the Dirichlet mean with a variance term and a KL-divergence regularizer to encourage an explicit 'I do not know' state when evidence is scarce. Empirical results on MNIST and CIFAR-10 show competitive accuracy and significantly improved uncertainty quantification, enabling better detection of out-of-distribution inputs and stronger robustness to adversarial perturbations. Overall, evidential deep learning provides a practical, principled framework for uncertainty estimation in deterministic neural networks by leveraging subjective logic and Dirichlet posteriors.

Abstract

Deterministic neural nets have been shown to learn effective predictors on a wide range of machine learning problems. However, as the standard approach is to train the network to minimize a prediction loss, the resultant model remains ignorant to its prediction confidence. Orthogonally to Bayesian neural nets that indirectly infer prediction uncertainty through weight uncertainties, we propose explicit modeling of the same using the theory of subjective logic. By placing a Dirichlet distribution on the class probabilities, we treat predictions of a neural net as subjective opinions and learn the function that collects the evidence leading to these opinions by a deterministic neural net from data. The resultant predictor for a multi-class classification problem is another Dirichlet distribution whose parameters are set by the continuous output of a neural net. We provide a preliminary analysis on how the peculiarities of our new loss function drive improved uncertainty estimation. We observe that our method achieves unprecedented success on detection of out-of-distribution queries and endurance against adversarial perturbations.

Figures (6)

• Figure 1: Classification of the rotated digit $1$ (at bottom) at different angles between $0$ and $180$ degrees. Left: The classification probability is calculated using the softmax function. Right: The classification probability and uncertainty are calculated using the proposed method.
• Figure 2: The change of accuracy with respect to the uncertainty threshold for EDL.
• Figure 3: Test accuracies (%) for MNIST and CIFAR5 datasets.
• Figure 4: Empirical CDF for the entropy of the predictive distributions on the notMNIST dataset (left) and samples from the last five categories of CIFAR10 dataset (right).
• Figure 5: Accuracy and entropy as a function of the adversarial perturbation $\epsilon$ on the MNIST dataset.