Robust Uncertainty Estimation under Distribution Shift via Difference Reconstruction

TL;DR

This paper tackles unreliable uncertainty estimates under distribution shift in deep learning by addressing information loss in reconstruction-based methods. It introduces Difference Reconstruction Uncertainty Estimation (DRUE), which uses two decoders attached to different layers to measure uncertainty via the discrepancy between reconstructions, with $U_D(\mathbf{x}) = \|\hat{\mathbf{x}} - \hat{\mathbf{x}}'\|_1$. The authors provide gradient-based and information-theoretic justifications and show empirically that DRUE yields superior AUC and AUPR for out-of-distribution detection on glaucoma imaging and other datasets. Overall, DRUE offers a robust, scalable framework for trustworthy AI under domain shifts and can be extended to other tasks and architectures.

Abstract

Estimating uncertainty in deep learning models is critical for reliable decision-making in high-stakes applications such as medical imaging. Prior research has established that the difference between an input sample and its reconstructed version produced by an auxiliary model can serve as a useful proxy for uncertainty. However, directly comparing reconstructions with the original input is degraded by information loss and sensitivity to superficial details, which limits its effectiveness. In this work, we propose Difference Reconstruction Uncertainty Estimation (DRUE), a method that mitigates this limitation by reconstructing inputs from two intermediate layers and measuring the discrepancy between their outputs as the uncertainty score. To evaluate uncertainty estimation in practice, we follow the widely used out-of-distribution (OOD) detection paradigm, where in-distribution (ID) training data are compared against datasets with increasing domain shift. Using glaucoma detection as the ID task, we demonstrate that DRUE consistently achieves superior AUC and AUPR across multiple OOD datasets, highlighting its robustness and reliability under distribution shift. This work provides a principled and effective framework for enhancing model reliability in uncertain environments.

Figures (4)

• Figure 1: Illustration of reconstruction-based uncertainty estimation in glaucoma detection. (Top) Conventional input–reconstruction difference often highlights irrelevant regions, such as blood vessels, due to information loss. (Bottom) DRUE compares two reconstructions from different intermediate layers, reducing this effect and focusing uncertainty on clinically meaningful regions, such as the optic disc and cup (the bright, round region).
• Figure 2: Pipeline of DRUE. The upper part represents the classification model $M$, which takes an input $\mathbf{x}$ and generates a prediction $\hat{y}$. Attached below are the two decoders $G_1$ and $G_0$.
• Figure 3: Visual results of input images, reconstructions from DRUE, and the corresponding uncertainty maps (normalized to [0, 1]).
• Figure 4: Distribution of DRUE uncertainty across datasets. Uncertainty increases with greater data shift.

Theorems & Definitions (1)

• Definition 1: DRUE uncertainty