Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Big Batch Bayesian Active Learning by Considering Predictive Probabilities

Sebastian W. Ober, Samuel Power, Tom Diethe, Henry B. Moss

TL;DR

The paper identifies that BatchBALD conflates epistemic and aleatoric uncertainty, which can lead to suboptimal batch selections. It introduces BBB-AL, a predictive-probability–driven acquisition that maximizes the entropy of predictive class probabilities across a batch, with a tractable closed form under Gaussian, independent-class assumptions and a general sample-based variant using Ledoit-Wolf shrinkage. Compared to BatchBALD, BBB-AL achieves better accuracy and is significantly faster for large batch sizes on CIFAR-10 with a ResNet-8 surrogate, demonstrating practical scalability and improved data efficiency. The approach provides a scalable alternative for batch Bayesian active learning that better isolates and leverages epistemic uncertainty in classification tasks.

Abstract

We observe that BatchBALD, a popular acquisition function for batch Bayesian active learning for classification, can conflate epistemic and aleatoric uncertainty, leading to suboptimal performance. Motivated by this observation, we propose to focus on the predictive probabilities, which only exhibit epistemic uncertainty. The result is an acquisition function that not only performs better, but is also faster to evaluate, allowing for larger batches than before.

Big Batch Bayesian Active Learning by Considering Predictive Probabilities

TL;DR

The paper identifies that BatchBALD conflates epistemic and aleatoric uncertainty, which can lead to suboptimal batch selections. It introduces BBB-AL, a predictive-probability–driven acquisition that maximizes the entropy of predictive class probabilities across a batch, with a tractable closed form under Gaussian, independent-class assumptions and a general sample-based variant using Ledoit-Wolf shrinkage. Compared to BatchBALD, BBB-AL achieves better accuracy and is significantly faster for large batch sizes on CIFAR-10 with a ResNet-8 surrogate, demonstrating practical scalability and improved data efficiency. The approach provides a scalable alternative for batch Bayesian active learning that better isolates and leverages epistemic uncertainty in classification tasks.

Abstract

We observe that BatchBALD, a popular acquisition function for batch Bayesian active learning for classification, can conflate epistemic and aleatoric uncertainty, leading to suboptimal performance. Motivated by this observation, we propose to focus on the predictive probabilities, which only exhibit epistemic uncertainty. The result is an acquisition function that not only performs better, but is also faster to evaluate, allowing for larger batches than before.
Paper Structure (11 sections, 15 equations, 2 figures)

This paper contains 11 sections, 15 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Bayesian active learning
  4. The promises and pitfalls of BatchBALD
  5. Active learning for classification through predictive probabilities
  6. Computational complexity
  7. Experiments
  8. Related work
  9. Conclusion & future work
  10. Expression for binary GP classification
  11. Sample-based acquisition

Figures (2)

  • Figure 1: The GP prediction along with BatchBALD and BBB-AL (ours) acquisition landscapes for $B=2$, where $x_1$ (the first point in the batch) is on the x-axis and $x_2$ (the second point) is on the y-axis. For BatchBALD, we see that the optimal acquisition is $x_1 = x_2 = 1$, whereas for BBB-AL we obtain $x_1 = 0$, $x_2 = 1$.
  • Figure 2: Accuracy on CIFAR-10 for BBB-AL and BatchBALD acquisition functions versus time for $B=1$, $B=10$, and $B=50$. The darker lines show the mean performance, whereas the lighter lines show individual runs.