Open Challenges on Fairness of Artificial Intelligence in Medical Imaging Applications
Enzo Ferrante, Rodrigo Echeveste
TL;DR
This chapter surveys open challenges in ensuring fairness for AI in medical imaging, highlighting bias sources across data, models, and people. It discusses how biased metrics, leveling-down risks, varying task difficulty, unseen populations, and non-demographic confounders complicate fairness audits, emphasizing that there are no definitive solutions yet. The authors review potential directions such as adversarially reweighted learning, epistemic-uncertainty weighting, counterfactual explanations, and unsupervised audits, while cautioning about their limitations. The work underscores the need for robust, intersectional, and globally informed fairness practices to support reliable, ethically sound deployment of medical imaging AI tools.
Abstract
Recently, the research community of computerized medical imaging has started to discuss and address potential fairness issues that may emerge when developing and deploying AI systems for medical image analysis. This chapter covers some of the pressing challenges encountered when doing research in this area, and it is intended to raise questions and provide food for thought for those aiming to enter this research field. The chapter first discusses various sources of bias, including data collection, model training, and clinical deployment, and their impact on the fairness of machine learning algorithms in medical image computing. We then turn to discussing open challenges that we believe require attention from researchers and practitioners, as well as potential pitfalls of naive application of common methods in the field. We cover a variety of topics including the impact of biased metrics when auditing for fairness, the leveling down effect, task difficulty variations among subgroups, discovering biases in unseen populations, and explaining biases beyond standard demographic attributes.