Deep Bayesian Active Learning with Image Data

TL;DR

The paper tackles label-efficient learning for high-dimensional image data by integrating Bayesian deep learning into active learning through Bayesian convolutional networks (BCNNs) and MC dropout to capture epistemic uncertainty. It introduces acquisition functions like BALD, Max Entropy, Variation Ratios, and Mean STD, showing how to approximate them with MC samples from the BCNN posterior and evaluating them on MNIST and the ISIC melanoma dataset. The findings indicate that uncertainty-aware acquisitions (BALD, Variation Ratios, Max Entropy) outperform random baselines and deterministic models, with BALD often offering the fastest or most robust improvements, and that the approach can rival semi-supervised methods while using far fewer unlabeled data. The work demonstrates practical, data-efficient active learning for image tasks, with potential impact on cost reduction in medical imaging and other high-dimensional domains, while highlighting computational trade-offs and avenues for future optimization.

Abstract

Even though active learning forms an important pillar of machine learning, deep learning tools are not prevalent within it. Deep learning poses several difficulties when used in an active learning setting. First, active learning (AL) methods generally rely on being able to learn and update models from small amounts of data. Recent advances in deep learning, on the other hand, are notorious for their dependence on large amounts of data. Second, many AL acquisition functions rely on model uncertainty, yet deep learning methods rarely represent such model uncertainty. In this paper we combine recent advances in Bayesian deep learning into the active learning framework in a practical way. We develop an active learning framework for high dimensional data, a task which has been extremely challenging so far, with very sparse existing literature. Taking advantage of specialised models such as Bayesian convolutional neural networks, we demonstrate our active learning techniques with image data, obtaining a significant improvement on existing active learning approaches. We demonstrate this on both the MNIST dataset, as well as for skin cancer diagnosis from lesion images (ISIC2016 task).

Figures (5)

• Figure 1: MNIST test accuracy as a function of number of acquired images from the pool set (up to 1000 images, using validation set size 100, and averaged over 3 repetitions). Four acquisition functions (BALD, Variation Ratios, Max Entropy, and Mean STD) are evaluated and compared to a Random acquisition function.
• Figure 2: Test accuracy as a function of number of acquired images for various acquisition functions, using both a Bayesian CNN (red) and a deterministic CNN (blue).
• Figure 3: MNIST test accuracy (two digit classification) as a function of number acquired images, compared to a current technique for active learning of image data: MBR zhu2003combining.
• Figure 4: Skin cancer (melanoma) example lesions from the ISIC 2016 melanoma diagnosis dataset. The two lesions on the left are benign (non-cancerous), while the two lesions on the right are malignant (cancerous).
• Figure 5: AUC (left) as well as the number of acquired positive examples (right) for both the BALD acquisition function as well as uniform acquisition function, on ISIC 2016 melanoma diagnosis dataset. Two random test splits are assessed (top and bottom), and on each test set the experiment was repeated three times with different random seeds (shown mean with standard error).