Generalized Regularized Evidential Deep Learning Models: Theory and Comprehensive Evaluation

TL;DR

This work identifies a fundamental learning limitation in evidential deep learning: gradients vanish for samples mapped to zero-evidence regions, hindering learning precisely where supervision is needed. It introduces Correct Evidence Regularization and a generalized activation framework (GRED), including Shifted Exponential Linear Units, to restore meaningful gradients across evidence regimes. The proposed approach yields improved generalization and calibrated uncertainty across classification, few-shot learning, adversarial scenarios, OOD detection, and vision restoration tasks, demonstrated on benchmarks like MNIST, CIFAR datasets, Tiny-ImageNet, and CodeFormer-based blind face restoration. By enabling learning from all samples and providing robust uncertainty estimates, GRED advances trustworthy uncertainty-aware neural networks with practical impact in safety-critical and data-constrained domains.

Abstract

Evidential deep learning (EDL) models, based on Subjective Logic, introduce a principled and computationally efficient way to make deterministic neural networks uncertainty-aware. The resulting evidential models can quantify fine-grained uncertainty using learned evidence. However, the Subjective-Logic framework constrains evidence to be non-negative, requiring specific activation functions whose geometric properties can induce activation-dependent learning-freeze behavior: a regime where gradients become extremely small for samples mapped into low-evidence regions. We theoretically characterize this behavior and analyze how different evidential activations influence learning dynamics. Building on this analysis, we design a general family of activation functions and corresponding evidential regularizers that provide an alternative pathway for consistent evidence updates across activation regimes. Extensive experiments on four benchmark classification problems (MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet), two few-shot classification problems, and blind face restoration problem empirically validate the developed theory and demonstrate the effectiveness of the proposed generalized regularized evidential models.

Figures (20)

• Figure 1: Cifar-100 Result
• Figure 2: Intuitive visualization of a zero-evidence region for evidential models in the evidence space for binary classification. Samples mapped into such regions have extremely small gradients, leading to limited model update during training. GRED encourages larger gradients for 'zero-evidence' samples, enabling consistent learning across samples.
• Figure 3: Graphical model for Evidential Models
• Figure 4: Output Evidence and Gradient plot of different evidential activations for different Logit values
• Figure 5: Toy dataset with 4 data points

Theorems & Definitions (8)

• Definition 1: Zero Evidence Sample and Zero Evidence Region
• Theorem 1
• proof
• Corollary 1
• Theorem 2
• proof
• Theorem 3
• proof