Epistemic Deep Learning: Enabling Machine Learning Models to Know When They Do Not Know

TL;DR

Epistemic Deep Learning reframes uncertainty in AI as a second-order concern, focusing on epistemic uncertainty arising from limited or biased data. It introduces Random-Set Neural Networks (RS-NN), a classifier that outputs belief functions over sets of classes via random-set theory, enabling set-valued predictions and quantified ignorance. A budgeting mechanism selects a tractable subset of focal sets, and a unified evaluation framework maps heterogeneous uncertainty outputs to credal sets for fair model comparison. The framework is extended to Random-Set Large Language Models (RS-LLMs) and applied to autonomous racing weather/cone classification, delivering improved OoD detection, calibration, and adversarial robustness while scaling to large architectures. Together, these developments advance Epistemic AI by enabling models to know when they do not know, with practical implications for safety-critical systems, NLP, and autonomous decision-making.

Abstract

Machine learning has achieved remarkable successes, yet its deployment in safety-critical domains remains hindered by an inherent inability to manage uncertainty, resulting in overconfident and unreliable predictions when models encounter out-of-distribution data, adversarial perturbations, or naturally fluctuating environments. This thesis, titled Epistemic Deep Learning: Enabling Machine Learning Models to 'Know When They Do Not Know', addresses these critical challenges by advancing the paradigm of Epistemic Artificial Intelligence, which explicitly models and quantifies epistemic uncertainty: the uncertainty arising from limited, biased, or incomplete training data, as opposed to the irreducible randomness of aleatoric uncertainty, thereby empowering models to acknowledge their limitations and refrain from overconfident decisions when uncertainty is high. Central to this work is the development of the Random-Set Neural Network (RS-NN), a novel methodology that leverages random set theory to predict belief functions over sets of classes, capturing the extent of epistemic uncertainty through the width of associated credal sets, applications of RS-NN, including its adaptation to Large Language Models (LLMs) and its deployment in weather classification for autonomous racing. In addition, the thesis proposes a unified evaluation framework for uncertainty-aware classifiers. Extensive experiments validate that integrating epistemic awareness into deep learning not only mitigates the risks associated with overconfident predictions but also lays the foundation for a paradigm shift in artificial intelligence, where the ability to 'know when it does not know' becomes a hallmark of robust and dependable systems. The title encapsulates the core philosophy of this work, emphasizing that true intelligence involves recognizing and managing the limits of one's own knowledge.

Figures (97)

• Figure 1: Overview of the thesis structure.
• Figure 2: Confidence scores of uncertainty-aware (Epistemic) and standard model with no uncertainty estimation (Traditional) for samples of ImageNet-A (adversarial) dataset. Traditional model is overconfident in a misclassification with confidence scores of 86.9% and 94.95%, whereas the epistemic model exhibits lower confidence with scores of 32.93% and 3.96% for misclassifications.
• Figure 3: In-distribution (left) vs. out-of-distribution (right) datasets.
• Figure 4: Major approaches to uncertainty in AI. While traditional networks and Deterministic uncertainty models mukhoti2023deep (a) have deterministic weights, and output either a determistic output value or a probability vector in the output space, Bayesian neural networks blundell2015weightgal2016dropoutrudner2022tractable (b) compute a predictive distribution there by integrating over a learnt posterior distribution of model parameters given training data. As this is often infeasible due to the complexity of the posterior, Bayesian Model Averaging (BMA) hinne2020conceptualgraefe2015limitations is often used to approximate the predictive distribution by averaging over predictions from multiple samples. The same (averaging) technique can be applied to (deep) ensembles of networks lakshminarayanan2017simple (c), in which a number of models are trained independently using different initializations. Evidential approaches sensoy (d) make predictions as parameters of a second-order Dirichlet distribution on the output space, instead of softmax probabilities. Finally, epistemic approaches (e) employ second-order probability representations, either in the output space or the target space, e.g. in the form of interval probabilities, credal sets or random sets manchingal2025randomsetneuralnetworksrsnn. A pignistic probability SMETS2005133 estimate can be obtained from credal sets for comparison to other models manchingal2025unifiedevaluationframeworkepistemic.
• Figure 5: Visualizations of 100 prediction samples obtained prior to Bayesian Model Averaging and corresponding Bayesian Model Averaged prediction in two real scenarios from CIFAR-10.