Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CUPID: A Plug-in Framework for Joint Aleatoric and Epistemic Uncertainty Estimation with a Single Model

Xinran Xu, Xiuyi Fan

TL;DR

CUPID (Comprehensive Uncertainty Plug-in estImation moDel), a general-purpose module that jointly estimates aleatoric and epistemic uncertainty without modifying or retraining the base model, and supports more transparent and trustworthy AI.

Abstract

Accurate estimation of uncertainty in deep learning is critical for deploying models in high-stakes domains such as medical diagnosis and autonomous decision-making, where overconfident predictions can lead to harmful outcomes. In practice, understanding the reason behind a model's uncertainty and the type of uncertainty it represents can support risk-aware decisions, enhance user trust, and guide additional data collection. However, many existing methods only address a single type of uncertainty or require modifications and retraining of the base model, making them difficult to adopt in real-world systems. We introduce CUPID (Comprehensive Uncertainty Plug-in estImation moDel), a general-purpose module that jointly estimates aleatoric and epistemic uncertainty without modifying or retraining the base model. CUPID can be flexibly inserted into any layer of a pretrained network. It models aleatoric uncertainty through a learned Bayesian identity mapping and captures epistemic uncertainty by analyzing the model's internal responses to structured perturbations. We evaluate CUPID across a range of tasks, including classification, regression, and out-of-distribution detection. The results show that it consistently delivers competitive performance while offering layer-wise insights into the origins of uncertainty. By making uncertainty estimation modular, interpretable, and model-agnostic, CUPID supports more transparent and trustworthy AI. Related code and data are available at https://github.com/a-Fomalhaut-a/CUPID.

CUPID: A Plug-in Framework for Joint Aleatoric and Epistemic Uncertainty Estimation with a Single Model

TL;DR

CUPID (Comprehensive Uncertainty Plug-in estImation moDel), a general-purpose module that jointly estimates aleatoric and epistemic uncertainty without modifying or retraining the base model, and supports more transparent and trustworthy AI.

Abstract

Accurate estimation of uncertainty in deep learning is critical for deploying models in high-stakes domains such as medical diagnosis and autonomous decision-making, where overconfident predictions can lead to harmful outcomes. In practice, understanding the reason behind a model's uncertainty and the type of uncertainty it represents can support risk-aware decisions, enhance user trust, and guide additional data collection. However, many existing methods only address a single type of uncertainty or require modifications and retraining of the base model, making them difficult to adopt in real-world systems. We introduce CUPID (Comprehensive Uncertainty Plug-in estImation moDel), a general-purpose module that jointly estimates aleatoric and epistemic uncertainty without modifying or retraining the base model. CUPID can be flexibly inserted into any layer of a pretrained network. It models aleatoric uncertainty through a learned Bayesian identity mapping and captures epistemic uncertainty by analyzing the model's internal responses to structured perturbations. We evaluate CUPID across a range of tasks, including classification, regression, and out-of-distribution detection. The results show that it consistently delivers competitive performance while offering layer-wise insights into the origins of uncertainty. By making uncertainty estimation modular, interpretable, and model-agnostic, CUPID supports more transparent and trustworthy AI. Related code and data are available at https://github.com/a-Fomalhaut-a/CUPID.
Paper Structure (65 sections, 3 theorems, 38 equations, 9 figures, 27 tables)

This paper contains 65 sections, 3 theorems, 38 equations, 9 figures, 27 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Model-redefining approaches
  4. Model-preserving approaches
  5. Method
  6. Problem Formulation
  7. Aleatoric Uncertainty Estimation with CUPID
  8. Epistemic Uncertainty Estimation with CUPID
  9. Loss Function
  10. Experiments
  11. Medical Image Misclassification Detection
  12. Experiment setting
  13. Results
  14. OOD Detection
  15. Experiment setting
  16. ...and 50 more sections

Key Result

Theorem 1

Assume that $F_l$ is locally differentiable at $\mathbf{m}_l$, and let $\Delta \mathbf{m}_l = \mathbf{m}_l' - \mathbf{m}_l$ be the reconstruction perturbation. Then, under a first-order Taylor approximation: where $J_{F_l}(\mathbf{m}_l)$ is the Jacobian of $F_l$ evaluated at $\mathbf{m}_l$.

Figures (9)

  • Figure 1: CUPID uncertainty estimation on a 1D regression toy problem. CUPID is inserted into an MLP-based predictive model. CUPID captures both aleatoric (blue) and epistemic (red) uncertainty.
  • Figure 2: The CUPID pipeline. Aleatoric uncertainty is estimated using a dedicated Uncertainty Branch, while epistemic uncertainty is captured by measuring the variance between the original model output $\hat{\mathbf{y}}$ and the perturbed output $\hat{\mathbf{y}}'$.
  • Figure 3: Comparison of visual results between error and uncertainty maps. CUPID Aleatoric shows the best texture alignment and highest correlation with error maps.
  • Figure 4: Performance of CUPID inserted at varying locations: misclassification detection (Left) and super-resolution (Right). Aleatoric uncertainty estimation improves when CUPID is placed closer to the output, while epistemic uncertainty benefits from earlier insertion points.
  • Figure 5: Data samples from GLV2 and HAM10000.
  • ...and 4 more figures

Theorems & Definitions (3)

  • Theorem 1: Sensitivity & Deviation Driven Approximation of Epistemic Uncertainty
  • Proposition 1
  • Proposition 2