Outlier Detection in Large Radiological Datasets using UMAP

TL;DR

The paper addresses the challenge of cleanly curating large radiology datasets plagued by labeling and image-quality errors. It presents a UMAP-based visual analytics pipeline that leverages DenseNet-121 features to project images into a 2D space where outliers form distinct satellite clusters, facilitating rapid data quality assessment. Through experiments on ChestX-ray14, CheXpert, and MURA, the approach identifies view-specific clusters, corrupted images, and mislabeled samples, while showing that pretraining and embedding choices influence outlier detectability. The work offers a practical, data-type-agnostic tool for dataset curation with broad implications for improving the reliability of medical imaging ML applications, and provides code for reproducibility.

Abstract

The success of machine learning algorithms heavily relies on the quality of samples and the accuracy of their corresponding labels. However, building and maintaining large, high-quality datasets is an enormous task. This is especially true for biomedical data and for meta-sets that are compiled from smaller ones, as variations in image quality, labeling, reports, and archiving can lead to errors, inconsistencies, and repeated samples. Here, we show that the uniform manifold approximation and projection (UMAP) algorithm can find these anomalies essentially by forming independent clusters that are distinct from the main (good) data but similar to other points with the same error type. As a representative example, we apply UMAP to discover outliers in the publicly available ChestX-ray14, CheXpert, and MURA datasets. While the results are archival and retrospective and focus on radiological images, the graph-based methods work for any data type and will prove equally beneficial for curation at the time of dataset creation.

Figures (10)

• Figure 1: Schematic of the outlier search algorithm. Image features extracted from a DenseNet-121 neural network are projected onto a low-dimensional space (2-D plane) using UMAP.
• Figure 2: Outlier detection in the ChestX-ray14 dataset. (a) 2-D embedding. Labeled clusters from (a) are: (b) Lateral x-rays which were not supposed to be in the dataset, (c) PA x-rays with borders, (d) AP x-rays with borders, and (e) cluster from a single patient.
• Figure 3: Outlier detection in the CheXpert dataset. (a) 2-D Embedding. Example images with (b) block artifacts, (c) noise, (d) improper dynamic range, (e) vertical artifacts, and (f) alignment issues.
• Figure 4: Embedding of CheXpert dataset using different pre-trained models. DenseNet-121 and ResNet-50 trained on ImageNet (left two) and ChestX-ray14 (right tow) datasets. Each yellow point represents an image with vertical artifact (from cluster e in Fig. \ref{['fig:CheXpert_complete']} (a)) indicating chest x-ray pre-trained models fail to identify these as outliers.
• Figure 6: Embedding of CheXpert dataset using several dimensionality reduction algorithms. (a) PCA, (b) t-SNE, (c) t-SNE (exaggerated), (d) TriMap, and (e) PaCMAP. Each yellow point represents an image with vertical artifacts (from cluster e in Fig. \ref{['fig:CheXpert_complete']} (a)).