Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Study of Acquisition Functions for Medical Imaging Deep Active Learning

Bonaventure F. P. Dossou

TL;DR

The paper tackles how different acquisition functions for deep active learning perform in a data-scarce medical imaging task. It employs Bayesian CNNs with MC-Dropout to quantify epistemic uncertainty and evaluates three criteria—BALD, Maximum Entropy, and MeanSTD—on the ISIC 2016 melanoma dataset. Key findings show BALD generally yields the best average performance, while MeanSTD is unstable and Maximum Entropy is robust to sampling mode, though all methods can exploit extreme class imbalance in very small data regimes. The work offers practical guidance for selecting acquisition strategies in low-label medical imaging settings and points to future directions including larger datasets and the EPIG approach.

Abstract

The Deep Learning revolution has enabled groundbreaking achievements in recent years. From breast cancer detection to protein folding, deep learning algorithms have been at the core of very important advancements. However, these modern advancements are becoming more and more data-hungry, especially on labeled data whose availability is scarce: this is even more prevalent in the medical context. In this work, we show how active learning could be very effective in data scarcity situations, where obtaining labeled data (or annotation budget is very limited). We compare several selection criteria (BALD, MeanSTD, and MaxEntropy) on the ISIC 2016 dataset. We also explored the effect of acquired pool size on the model's performance. Our results suggest that uncertainty is useful to the Melanoma detection task, and confirms the hypotheses of the author of the paper of interest, that \textit{bald} performs on average better than other acquisition functions. Our extended analyses however revealed that all acquisition functions perform badly on the positive (cancerous) samples, suggesting exploitation of class unbalance, which could be crucial in real-world settings. We finish by suggesting future work directions that would be useful to improve this current work. The code of our implementation is open-sourced at \url{https://github.com/bonaventuredossou/ece526_course_project}

A Study of Acquisition Functions for Medical Imaging Deep Active Learning

TL;DR

The paper tackles how different acquisition functions for deep active learning perform in a data-scarce medical imaging task. It employs Bayesian CNNs with MC-Dropout to quantify epistemic uncertainty and evaluates three criteria—BALD, Maximum Entropy, and MeanSTD—on the ISIC 2016 melanoma dataset. Key findings show BALD generally yields the best average performance, while MeanSTD is unstable and Maximum Entropy is robust to sampling mode, though all methods can exploit extreme class imbalance in very small data regimes. The work offers practical guidance for selecting acquisition strategies in low-label medical imaging settings and points to future directions including larger datasets and the EPIG approach.

Abstract

The Deep Learning revolution has enabled groundbreaking achievements in recent years. From breast cancer detection to protein folding, deep learning algorithms have been at the core of very important advancements. However, these modern advancements are becoming more and more data-hungry, especially on labeled data whose availability is scarce: this is even more prevalent in the medical context. In this work, we show how active learning could be very effective in data scarcity situations, where obtaining labeled data (or annotation budget is very limited). We compare several selection criteria (BALD, MeanSTD, and MaxEntropy) on the ISIC 2016 dataset. We also explored the effect of acquired pool size on the model's performance. Our results suggest that uncertainty is useful to the Melanoma detection task, and confirms the hypotheses of the author of the paper of interest, that \textit{bald} performs on average better than other acquisition functions. Our extended analyses however revealed that all acquisition functions perform badly on the positive (cancerous) samples, suggesting exploitation of class unbalance, which could be crucial in real-world settings. We finish by suggesting future work directions that would be useful to improve this current work. The code of our implementation is open-sourced at \url{https://github.com/bonaventuredossou/ece526_course_project}
Paper Structure (9 sections, 7 equations, 12 figures, 3 tables)

This paper contains 9 sections, 7 equations, 12 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Acquisition Functions
  3. Maximum Entropy
  4. Mean Standard Deviation
  5. BALD
  6. Dataset and Task Description
  7. Hyper-parameters and Experiments
  8. Results and Discussion
  9. Conclusion and Future Works

Figures (12)

  • Figure 1: Picture of a non-cancerous Skin (Negative Sample)
  • Figure 2: Picture of a cancerous Skin (Positive Sample)
  • Figure 3: Repartition of Classes across Training, Validation, and Testing splits
  • Figure 4: Evaluation Results of Bayesian CNNs with and without uncertainty
  • Figure 5: Evaluation Results of Bayesian CNNs across Active Learning Rounds (from top to bottom)
  • ...and 7 more figures