Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comprehensive Survey on Evidential Deep Learning and Its Applications

Junyu Gao, Mengyuan Chen, Liangyu Xiang, Changsheng Xu

TL;DR

The paper addresses the need for reliable uncertainty estimation in deep learning without prohibitive computational costs. It surveys Evidential Deep Learning (EDL), a single-forward-pass framework grounded in subjective logic and Dirichlet posteriors, detailing its theoretical foundations, methodological expansions, and wide-ranging applications. The contributions include a structured taxonomy of EDL developments (evidence collection, OOD handling, training strategies, and regression), a comprehensive review of EDL-enabled ML paradigms, and practical guidance for deployment in open-world and scientific settings. The work highlights EDL's potential to deliver calibrated, interpretable uncertainty with low overhead, while outlining open questions and future directions for integrating EDL with foundation models and broader domains.

Abstract

Reliable uncertainty estimation has become a crucial requirement for the industrial deployment of deep learning algorithms, particularly in high-risk applications such as autonomous driving and medical diagnosis. However, mainstream uncertainty estimation methods, based on deep ensembling or Bayesian neural networks, generally impose substantial computational overhead. To address this challenge, a novel paradigm called Evidential Deep Learning (EDL) has emerged, providing reliable uncertainty estimation with minimal additional computation in a single forward pass. This survey provides a comprehensive overview of the current research on EDL, designed to offer readers a broad introduction to the field without assuming prior knowledge. Specifically, we first delve into the theoretical foundation of EDL, the subjective logic theory, and discuss its distinctions from other uncertainty estimation frameworks. We further present existing theoretical advancements in EDL from four perspectives: reformulating the evidence collection process, improving uncertainty estimation via OOD samples, delving into various training strategies, and evidential regression networks. Thereafter, we elaborate on its extensive applications across various machine learning paradigms and downstream tasks. In the end, an outlook on future directions for better performances and broader adoption of EDL is provided, highlighting potential research avenues.

A Comprehensive Survey on Evidential Deep Learning and Its Applications

TL;DR

The paper addresses the need for reliable uncertainty estimation in deep learning without prohibitive computational costs. It surveys Evidential Deep Learning (EDL), a single-forward-pass framework grounded in subjective logic and Dirichlet posteriors, detailing its theoretical foundations, methodological expansions, and wide-ranging applications. The contributions include a structured taxonomy of EDL developments (evidence collection, OOD handling, training strategies, and regression), a comprehensive review of EDL-enabled ML paradigms, and practical guidance for deployment in open-world and scientific settings. The work highlights EDL's potential to deliver calibrated, interpretable uncertainty with low overhead, while outlining open questions and future directions for integrating EDL with foundation models and broader domains.

Abstract

Reliable uncertainty estimation has become a crucial requirement for the industrial deployment of deep learning algorithms, particularly in high-risk applications such as autonomous driving and medical diagnosis. However, mainstream uncertainty estimation methods, based on deep ensembling or Bayesian neural networks, generally impose substantial computational overhead. To address this challenge, a novel paradigm called Evidential Deep Learning (EDL) has emerged, providing reliable uncertainty estimation with minimal additional computation in a single forward pass. This survey provides a comprehensive overview of the current research on EDL, designed to offer readers a broad introduction to the field without assuming prior knowledge. Specifically, we first delve into the theoretical foundation of EDL, the subjective logic theory, and discuss its distinctions from other uncertainty estimation frameworks. We further present existing theoretical advancements in EDL from four perspectives: reformulating the evidence collection process, improving uncertainty estimation via OOD samples, delving into various training strategies, and evidential regression networks. Thereafter, we elaborate on its extensive applications across various machine learning paradigms and downstream tasks. In the end, an outlook on future directions for better performances and broader adoption of EDL is provided, highlighting potential research avenues.
Paper Structure (33 sections, 37 equations, 5 figures, 1 table)

This paper contains 33 sections, 37 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Common Terms of Uncertainty Categories
  4. Existing Uncertainty Quantification Methods
  5. Theoretical Foundations of EDL
  6. Introduction of Subjective Logic (SL)
  7. Other Uncertainty Reasoning Frameworks
  8. Theoretical Explorations of EDL
  9. Evidential Deep Learning
  10. Reformulating Evidence Collection Process
  11. Improve Uncertainty Estimation via OOD samples
  12. Delving into Different Training Strategies
  13. Evidential Regression Network
  14. APPLICATIONS
  15. EDL Enhanced Machine Learning
  16. ...and 18 more sections

Figures (5)

  • Figure 1: Main categories of existing uncertainty quantification methods, including: single deterministic methods, Bayesian methods, ensemble methods, and post-hoc methods.
  • Figure 2: In 3-class classification, we present examples of Dirichlet distributions with their concentration parameters and subjective opinions across four different scenarios: (a) Dominant and Certain, (b) Dominant and Uncertain, (c) Balanced and Certain, and (d) Balanced and Uncertain.
  • Figure 3: Structure of theoretical explorations of evidential deep learning, including: (1) Reformulating evidence collection process ($^1$li2024hyper, $^2$deng2023uncertainty, $^3$pandey2023learn, $^4$shen2023postgao2023vectorized); (2) Improving uncertainty estimation via OOD samples ($^5$nagahama2023learningzhao2019quantifying, $^6$sensoy2020uncertaintyhu2021multidimensionaldavies2023knowledge); (3) Delving into different training strategies ($^7$li2022tedl, $^8$xia2022hybrid, $^9$sensoy2021misclassification, $^{10}$pandey2022multidimensionalchen2022dual,$^{11}$kandemir2022evidentialhaussmann2019bayesian); (4) Evidential regression network ($^{12}$amini2020deep, $^{13}$meinert2021multivariate, $^{14}$ma2021trustworthy, $^{15}$ye2024uncertaintywu2024evidenceoh2022improving, $^{16}$meinert2023unreasonableduan2024evidentialpandey2023evidential).
  • Figure 4: An overview of EDL-enhanced machine learning algorithms, including: weakly-supervised learning (section \ref{['WS']}), transfer learning (section \ref{['TL']}), active learning (section \ref{['AL']}), multi-view classification (section \ref{['MVC']}), multi-label learning (section \ref{['ML']}), reinforcement learning (section \ref{['RL']}), and graph neural networks (section \ref{['GNN']}).
  • Figure 5: An overview of downstream applications of EDL, including six fields: computer vision (section \ref{['CV']}), natural language processing (section \ref{['NLP']}), cross-modal learning (section \ref{['CM']}), automatic driving (section \ref{['AD']}), EDL in the open-world (section \ref{['OS']}) and EDL for science (section \ref{['SCI']})