Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Effect of Enforcing Fairness on Reshaping Explanations in Machine Learning Models

Joshua Wolff Anderson, Shyam Visweswaran

TL;DR

It is found that increasing model fairness across racial subgroups can significantly alter feature importance rankings, sometimes in different ways across groups, highlighting the need to jointly consider accuracy, fairness, and explainability in model assessment rather than in isolation.

Abstract

Trustworthy machine learning in healthcare requires strong predictive performance, fairness, and explanations. While it is known that improving fairness can affect predictive performance, little is known about how fairness improvements influence explainability, an essential ingredient for clinical trust. Clinicians may hesitate to rely on a model whose explanations shift after fairness constraints are applied. In this study, we examine how enhancing fairness through bias mitigation techniques reshapes Shapley-based feature rankings. We quantify changes in feature importance rankings after applying fairness constraints across three datasets: pediatric urinary tract infection risk, direct anticoagulant bleeding risk, and recidivism risk. We also evaluate multiple model classes on the stability of Shapley-based rankings. We find that increasing model fairness across racial subgroups can significantly alter feature importance rankings, sometimes in different ways across groups. These results highlight the need to jointly consider accuracy, fairness, and explainability in model assessment rather than in isolation.

The Effect of Enforcing Fairness on Reshaping Explanations in Machine Learning Models

TL;DR

It is found that increasing model fairness across racial subgroups can significantly alter feature importance rankings, sometimes in different ways across groups, highlighting the need to jointly consider accuracy, fairness, and explainability in model assessment rather than in isolation.

Abstract

Trustworthy machine learning in healthcare requires strong predictive performance, fairness, and explanations. While it is known that improving fairness can affect predictive performance, little is known about how fairness improvements influence explainability, an essential ingredient for clinical trust. Clinicians may hesitate to rely on a model whose explanations shift after fairness constraints are applied. In this study, we examine how enhancing fairness through bias mitigation techniques reshapes Shapley-based feature rankings. We quantify changes in feature importance rankings after applying fairness constraints across three datasets: pediatric urinary tract infection risk, direct anticoagulant bleeding risk, and recidivism risk. We also evaluate multiple model classes on the stability of Shapley-based rankings. We find that increasing model fairness across racial subgroups can significantly alter feature importance rankings, sometimes in different ways across groups. These results highlight the need to jointly consider accuracy, fairness, and explainability in model assessment rather than in isolation.

Paper Structure

This paper contains 7 sections, 2 equations, 3 figures, 2 tables.

Table of Contents

  1. Abstract
  2. Introduction
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements

Figures (3)

  • Figure 1: Example local and global explanations for predicting urinary tract infection (UTI) in pediatric patients. (a) Waterfall plot for a patient in the UTI dataset, showing SHAP feature importance values corresponding to the patient's predicted probability of having a UTI. (b) Bar plot showing the mean absolute SHAP values for each feature in the UTI dataset, demonstrating the ranking and relative influence of features on the model's predicted probability of UTI in pediatric patients.
  • Figure 2: EOD values of four models evaluated on the UTI, COMPAS, and AFib test datasets before and after bias mitigation.
  • Figure 3: Feature importance of the seven features in the UTI dataset, as measured by global SHAP values, is shown for both baseline and bias mitigated models. Importance is computed separately for the two racial groups.