A conformalized learning of a prediction set with applications to medical imaging classification

TL;DR

The paper addresses uncertainty quantification in medical imaging classification by producing prediction sets with guaranteed coverage at a user-specified level $1-\alpha$. It introduces the Conformalized Prediction Set Network (CPSN), which learns an instance-specific threshold via a regression network to predict the Adaptive Prediction Score and then applies a conformalization step using validation residuals to adjust the threshold to $q(x)+\delta(x)$. Compared with standard conformal prediction variants (APS, RAPS), CPSN achieves smaller, yet reliable, prediction sets on OrganAMNIST and TissueMNIST while maintaining the target coverage, demonstrating improved efficiency without compromising safety. The method is presented as general and adaptable to other classification tasks beyond medical imaging, potentially enhancing clinical decision support by reducing uncertainty in predictions.

Abstract

Medical imaging classifiers can achieve high predictive accuracy, but quantifying their uncertainty remains an unresolved challenge, which prevents their deployment in medical clinics. We present an algorithm that can modify any classifier to produce a prediction set containing the true label with a user-specified probability, such as 90%. We train a network to predict an instance-based version of the Conformal Prediction threshold. The threshold is then conformalized to ensure the required coverage. We applied the proposed algorithm to several standard medical imaging classification datasets. The experimental results demonstrate that our method outperforms current approaches in terms of smaller average size of the prediction set while maintaining the desired coverage.