Towards Out-of-Distribution Detection for breast cancer classification in Point-of-Care Ultrasound Imaging

TL;DR

The results show that the energy score method outperforms the softmax method, performing well on two of the data sets, and the ensemble method is the most robust, performing the best at detecting OOD samples for all three OOD data sets.

Abstract

Deep learning has shown to have great potential in medical applications. In critical domains as such, it is of high interest to have trustworthy algorithms which are able to tell when reliable assessments cannot be guaranteed. Detecting out-of-distribution (OOD) samples is a crucial step towards building a safe classifier. Following a previous study, showing that it is possible to classify breast cancer in point-of-care ultrasound images, this study investigates OOD detection using three different methods: softmax, energy score and deep ensembles. All methods are tested on three different OOD data sets. The results show that the energy score method outperforms the softmax method, performing well on two of the data sets. The ensemble method is the most robust, performing the best at detecting OOD samples for all three OOD data sets.

Figures (6)

• Figure 1: POCUS images capturing normal tissue, benign and malignant lesions (from the left to right).
• Figure 2: Example of a POCUS image of poor quality, and images from the OOD data sets MNIST, CorruptPOCUS and CCA (left to right).
• Figure 3: Architecture of CNN with additional classifiers and exits. The scheme was created in NN-SVG LeNail2019.
• Figure 4: ROC curves for the OOD detection methods evaluated on MNIST (left), CorruptPOCUS (middle) and CCA (right).
• Figure 5: Distribution of energy scores for the OOD data sets. Energies from exit 1 (top), exit 2 (middle) and exit 3 (bottom). The vertical line marks the threshold where 95% of the POCUS test set images are classified as ID.