Adaptive Label Error Detection: A Bayesian Approach to Mislabeled Data Detection

TL;DR

The paper tackles mislabeled data in medical imaging by introducing Adaptive Label Error Detection (ALED), a data-centric approach that operates on a denoised, intermediate feature space of a deep CNN. ALED models each class in this reduced feature space with Gaussian distributions estimated via robust MCD, and uses an ensemble of random projections to perform a Bayesian likelihood-ratio test for mislabeling. Compared to Confident Learning baselines, ALED delivers higher sensitivity while maintaining precision, and cleaning data with ALED leads to substantial downstream gains, including a 33.8% reduction in test error for pretrained networks. The method is implemented in the statlab Python package and demonstrates robust performance across multiple architectures and four medical imaging datasets, highlighting the practical impact of improving data quality in deep learning pipelines.

Abstract

Machine learning classification systems are susceptible to poor performance when trained with incorrect ground truth labels, even when data is well-curated by expert annotators. As machine learning becomes more widespread, it is increasingly imperative to identify and correct mislabeling to develop more powerful models. In this work, we motivate and describe Adaptive Label Error Detection (ALED), a novel method of detecting mislabeling. ALED extracts an intermediate feature space from a deep convolutional neural network, denoises the features, models the reduced manifold of each class with a multidimensional Gaussian distribution, and performs a simple likelihood ratio test to identify mislabeled samples. We show that ALED has markedly increased sensitivity, without compromising precision, compared to established label error detection methods, on multiple medical imaging datasets. We demonstrate an example where fine-tuning a neural network on corrected data results in a 33.8% decrease in test set errors, providing strong benefits to end users. The ALED detector is deployed in the Python package statlab.

Figures (6)

• Figure 1: Mislabeled data detection using pretrained DenseNet121 trained on 5$\%$ mislabeled data over all four datasets.
• Figure 2: Mislabeled data detection using pretrained ResNet50 trained on 5$\%$ mislabeled data over all four datasets.
• Figure 3: Mislabeled data detection using randomly initialized DenseNet121 trained on 5$\%$ mislabeled data over all four datasets.
• Figure 4: Mislabeled data detection using randomly initialized ResNet50 trained on 5$\%$ mislabeled data over all four datasets.
• Figure 5: Mislabeled data detection on PneumoniaMNIST using pretrained ResNet50 trained over a range of mislabeling rates.