Fair Diagnosis: Leveraging Causal Modeling to Mitigate Medical Bias

TL;DR

This work tackles fairness in medical imaging by formalizing a causal framework that distinguishes direct bias from genuine clinical signals. It introduces Diagnosis Fairness (DF) and Approximate Diagnosis Fairness (ADF) and implements them via data-utility pretraining and adversarial masking to suppress the direct effect of sensitive attributes on predictions. The approach leverages path-specific causality and mutual-information objectives to preserve diagnostic performance while reducing bias, demonstrated across MIMIC-CXR, CheXpert, and TCGA-LUAD with GradCAM-based explainability. Overall, the method achieves fairness improvements with minimal sacrifice to accuracy, highlighting the potential of causal reasoning for fair, interpretable AI in clinical decision support.

Abstract

In medical image analysis, model predictions can be affected by sensitive attributes, such as race and gender, leading to fairness concerns and potential biases in diagnostic outcomes. To mitigate this, we present a causal modeling framework, which aims to reduce the impact of sensitive attributes on diagnostic predictions. Our approach introduces a novel fairness criterion, \textbf{Diagnosis Fairness}, and a unique fairness metric, leveraging path-specific fairness to control the influence of demographic attributes, ensuring that predictions are primarily informed by clinically relevant features rather than sensitive attributes. By incorporating adversarial perturbation masks, our framework directs the model to focus on critical image regions, suppressing bias-inducing information. Experimental results across multiple datasets demonstrate that our framework effectively reduces bias directly associated with sensitive attributes while preserving diagnostic accuracy. Our findings suggest that causal modeling can enhance both fairness and interpretability in AI-powered clinical decision support systems.

Figures (4)

• Figure 1: The SCM of our framework, where arrows represent causal paths. Here, $S$ denotes the sensitive attribute, $Y$ represents the true diagnosis, $\hat{Y}$ is the predicted diagnosis, and $X$ is the input modality (e.g., X-ray images). Red paths indicate biased paths that will be removed to improve diagnosis fairness.
• Figure 2: Overview of the Fair Diagnosis framework. The deployed models, $f_\theta$ and $f_h$ (in grey), are fixed after pretraining. Data utility enhancement (in blue) is applied during pretraining, while adversarial training (in orange) ensures fair masking and maintains original model performance.
• Figure 3: Data utility evaluations on MIMIC-CXR dataset, showing the fine-tuning performance of $f_h$ across eight different downstream tasks.
• Figure 4: Explainability analysis on both MIMIC-CXR and TCGA-LUAD datasets. GradCAM selvaraju2017gradcam is applied on the last convolution layer of $f_\theta$ when applied on $\hat{Y}$, and on $\mathbf{d}_Y$ when applied on $\hat{S}$.

Theorems & Definitions (4)

• Theorem 1
• proof
• Theorem 2
• proof