Bias and Generalizability of Foundation Models across Datasets in Breast Mammography
Elodie Germani, Ilayda Selin Türk, Fatima Zeineddine, Charbel Mourad, Shadi Albarqouni
TL;DR
This work addresses cross-dataset bias and generalizability of foundation models for breast mammography, demonstrating that modality-specific pre-training can boost in-domain accuracy but fails to generalize across diverse data sources. By aggregating a broad collection of datasets, including LBMD from Lebanon, the study shows generalization improves yet biases persist, especially for under-represented subgroups, and domain-adaptation methods often trade off overall performance for fairness. Fairness-aware strategies (e.g., GroupDRO) yield more stable and equitable performance across subgroups, while domain-adversarial methods (e.g., DANN) can reduce bias at the cost of accuracy. The findings underscore the need for fairness-aware FM training and evaluation, potentially in conjunction with federated learning, to develop inclusive, generalizable medical AI tools for radiology.
Abstract
Over the past decades, computer-aided diagnosis tools for breast cancer have been developed to enhance screening procedures, yet their clinical adoption remains challenged by data variability and inherent biases. Although foundation models (FMs) have recently demonstrated impressive generalizability and transfer learning capabilities by leveraging vast and diverse datasets, their performance can be undermined by spurious correlations that arise from variations in image quality, labeling uncertainty, and sensitive patient attributes. In this work, we explore the fairness and bias of FMs for breast mammography classification by leveraging a large pool of datasets from diverse sources-including data from underrepresented regions and an in-house dataset. Our extensive experiments show that while modality-specific pre-training of FMs enhances performance, classifiers trained on features from individual datasets fail to generalize across domains. Aggregating datasets improves overall performance, yet does not fully mitigate biases, leading to significant disparities across under-represented subgroups such as extreme breast densities and age groups. Furthermore, while domain-adaptation strategies can reduce these disparities, they often incur a performance trade-off. In contrast, fairness-aware techniques yield more stable and equitable performance across subgroups. These findings underscore the necessity of incorporating rigorous fairness evaluations and mitigation strategies into FM-based models to foster inclusive and generalizable AI.