Deep Sub-Ensembles for Fast Uncertainty Estimation in Image Classification

TL;DR

The paper tackles the computational burden of Deep Ensembles for uncertainty estimation in image classification by introducing Deep Sub-Ensembles, which share a trunk and ensemble only the upper layers. This design reduces inference cost while preserving competitive uncertainty metrics, achieving up to 1.5–2.5x speedups on CIFAR-10 and 5–15x on SVHN, with modest increases in error and similar NLL depending on the dataset and ensemble depth. Across MNIST, CIFAR-10, and SVHN, SEs approximate DEs with a tunable trade-off between accuracy and uncertainty quality, and calibration remains robust as more ensemble members are added. The work also analyzes OOD detection and the influence of the trunk, showing practical benefits for fast uncertainty estimates in robotics and related applications, with future work aimed at larger datasets and end-to-end training.

Abstract

Fast estimates of model uncertainty are required for many robust robotics applications. Deep Ensembles provides state of the art uncertainty without requiring Bayesian methods, but still it is computationally expensive. In this paper we propose deep sub-ensembles, an approximation to deep ensembles where the core idea is to ensemble only the layers close to the output, and not the whole model. With ResNet-20 on the CIFAR10 dataset, we obtain 1.5-2.5 speedup over a Deep Ensemble, with a small increase in error and NLL, and similarly up to 5-15 speedup with a VGG-like network on the SVHN dataset. Our results show that this idea enables a trade-off between error and uncertainty quality versus computational performance.

Figures (8)

• Figure 1: Conceptual comparison of Deep Ensembles and Deep Sub-Ensembles with $n$ ensemble members. The figure shows that in the latter, only a single trunk network $T_f$ is shared across all ensemble members, while in the former multiple trunk networks $T_i$ are used. In both cases the ensemble predictions are combined to produce outputs with uncertainty.
• Figure 2: Training and Inference process for Deep Sub-Ensembles
• Figure 3: Results on MNIST (with a simple CNN) and CIFAR10 (with ResNet-20), showing error and negative log-likelihood as the number of ensembles is varied
• Figure 4: Results on SVHN using a batch normalized VGG-like network
• Figure 5: Relationship between Sub-Ensemble and trunk network performance, in terms of error and negative log-likelihood. Here we only evaluate SE-1.