Detecting algorithmic bias in medical-AI models using trees

TL;DR

This paper presents an innovative framework for detecting areas of algorithmic bias in medical-AI decision support systems by employing the Classification and Regression Trees (CART) algorithm with conformity scores in the context of sepsis prediction.

Abstract

With the growing prevalence of machine learning and artificial intelligence-based medical decision support systems, it is equally important to ensure that these systems provide patient outcomes in a fair and equitable fashion. This paper presents an innovative framework for detecting areas of algorithmic bias in medical-AI decision support systems. Our approach efficiently identifies potential biases in medical-AI models, specifically in the context of sepsis prediction, by employing the Classification and Regression Trees (CART) algorithm with conformity scores. We verify our methodology by conducting a series of synthetic data experiments, showcasing its ability to estimate areas of bias in controlled settings precisely. The effectiveness of the concept is further validated by experiments using electronic medical records from Grady Memorial Hospital in Atlanta, Georgia. These tests demonstrate the practical implementation of our strategy in a clinical environment, where it can function as a vital instrument for guaranteeing fairness and equity in AI-based medical decisions.

Figures (10)

• Figure 1: Example of an optimal axis-aligned decision tree with a depth of $K=2$ with $p=2$ dimensions. Splits occur along specific features in the form $x_j = b$ for $j = 1,2$.
• Figure 2: Illustration of the algorithmic bias region $\mathcal{S}$ in the feature space, where the algorithm $\mathcal{A}$ exhibits suboptimal performance.
• Figure 3: The plots present 2D examples of (a) the determination of no bias, and (b) the determination of bias when using the conformal prediction procedure within our bias detection framework.
• Figure 4: Examples of the experimental results in 2(a) and 3(b) dimensional space.
• Figure 5: The plots show the mean coverage ratio for multiple $n$-dimensional test points: 2D(a), 3D(b), 4D(c), and 5D(d).