Post-hoc Orthogonalization for Mitigation of Protected Feature Bias in CXR Embeddings

TL;DR

The paper investigates bias in chest X-ray embeddings by protected features (age, sex, race) and introduces a post-hoc orthogonalization method to remove their influence. By projecting embeddings onto the space orthogonal to protected attributes, the authors produce $ ilde{E}$ that no longer encodes information about $X$, and they demonstrate this on MIMIC and CheXpert using three pre-trained embeddings. Results show that protected features significantly affect pathology predictions without orthogonalization, but after orthogonalization the protected-information predictability drops to near random and subgroup disparities diminish, with minimal to moderate impact on downstream pathology performance. This approach is model-agnostic and post-hoc, offering a practical fairness tool for medical imaging embeddings, while highlighting limitations such as linearity assumptions and residual disparities that motivate future work with nonlinear extensions and semi-structured models.

Abstract

Purpose: To analyze and remove protected feature effects in chest radiograph embeddings of deep learning models. Methods: An orthogonalization is utilized to remove the influence of protected features (e.g., age, sex, race) in CXR embeddings, ensuring feature-independent results. To validate the efficacy of the approach, we retrospectively study the MIMIC and CheXpert datasets using three pre-trained models, namely a supervised contrastive, a self-supervised contrastive, and a baseline classifier model. Our statistical analysis involves comparing the original versus the orthogonalized embeddings by estimating protected feature influences and evaluating the ability to predict race, age, or sex using the two types of embeddings. Results: Our experiments reveal a significant influence of protected features on predictions of pathologies. Applying orthogonalization removes these feature effects. Apart from removing any influence on pathology classification, while maintaining competitive predictive performance, orthogonalized embeddings further make it infeasible to directly predict protected attributes and mitigate subgroup disparities. Conclusion: The presented work demonstrates the successful application and evaluation of the orthogonalization technique in the domain of chest X-ray image classification.

Figures (7)

• Figure 1: Geometric visualization of the orthogonalization method. The column space of the protected features ($\text{col}(\bm X)$) contains all the possible vectors that can be formed by taking linear combinations of the respective features, i.e. the hypothesis space of a linear model. For an embedding vector $\bm e \in \bm E$, the orthogonalization is equivalent to the residual between $\bm e$ and its projection onto $\text{col}(\bm X)$. With $\mathcal{P}_X^\bot \bm e$ being perpendicular to $\text{col}(\bm X)$, the influence of protected features in $\bm e$ is neutralized.
• Figure 2: Marginalized distributions per sex (left two columns) and label Pleural Effusion (right two columns) over the PCA reduction of the original versus the orthogonalized corrected embedding.
• Figure 3: Regression/Classification performance for deriving protected features from an embedding vector with mean and standard deviation over 10 randomly initialized runs. The displayed metrics include mean absolute error (MAE) in years for age regression as well as AUC for classification of sex and race.
• Figure 4: Difference from overall AUC per subgroup based on downstream classifiers for original and orthogonalized embeddings. The closer a bar is to zero, the less disparity from the mean AUC exists.
• Figure S.1: Distribution of derived coefficients and p-values for 10 downstream models per embedding and protected feature category on the label Pleural Effusion.