Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Discrimination in machine learning algorithms

Roberta Pappadà, Francesco Pauli

TL;DR

This work tackles discrimination in data-driven decisions by employing causal-inference tools, with an emphasis on data preprocessing to detect and mitigate bias. It introduces a Coarsened Exact Matching (CEM) based discrimination score, $D_i$, and compares it to a kNN-based measure, $\delta_i$, using sequential CEM across random variable orders to stabilize estimates. The method is validated on real-world datasets (Adult, COMPAS, Custody) and through simulations that manipulate discrimination presence and conditioning variables, showing that the CEM-based measure can robustly detect discrimination and, in some settings, outperform the alternative. Overall, the approach provides a practical, auditing-oriented framework for assessing and reducing unfair treatment in high-stakes decisions.

Abstract

Machine learning algorithms are routinely used for business decisions that may directly affect individuals, for example, because a credit scoring algorithm refuses them a loan. It is then relevant from an ethical (and legal) point of view to ensure that these algorithms do not discriminate based on sensitive attributes (like sex or race), which may occur unwittingly and unknowingly by the operator and the management. Statistical tools and methods are then required to detect and eliminate such potential biases.

Discrimination in machine learning algorithms

TL;DR

This work tackles discrimination in data-driven decisions by employing causal-inference tools, with an emphasis on data preprocessing to detect and mitigate bias. It introduces a Coarsened Exact Matching (CEM) based discrimination score, , and compares it to a kNN-based measure, , using sequential CEM across random variable orders to stabilize estimates. The method is validated on real-world datasets (Adult, COMPAS, Custody) and through simulations that manipulate discrimination presence and conditioning variables, showing that the CEM-based measure can robustly detect discrimination and, in some settings, outperform the alternative. Overall, the approach provides a practical, auditing-oriented framework for assessing and reducing unfair treatment in high-stakes decisions.

Abstract

Machine learning algorithms are routinely used for business decisions that may directly affect individuals, for example, because a credit scoring algorithm refuses them a loan. It is then relevant from an ethical (and legal) point of view to ensure that these algorithms do not discriminate based on sensitive attributes (like sex or race), which may occur unwittingly and unknowingly by the operator and the management. Statistical tools and methods are then required to detect and eliminate such potential biases.
Paper Structure (11 sections, 6 equations, 11 figures)

This paper contains 11 sections, 6 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Measuring and avoiding discrimination by causal inference
  3. CEM based discrimination measure
  4. Testing the procedure
  5. The data
  6. The adult dataset.
  7. The COMPAS dataset.
  8. The Custody dataset.
  9. Removing and adding discrimination
  10. Introducing a conditioning variable correlated to S
  11. Conclusions

Figures (11)

  • Figure 1: $Y$ is the outcome, $S$ is the sensitive attribute, $X_i$ denotes observed variables, $Z$ denotes an unobserved variable. Example: $S$: race, $Y$: restitution of a loan, $X_1$: socioeconomic status, $X_2$: zip code residence, $Z$ availability of family financial support.
  • Figure 2: Pseudo-code for repeated sequential implementation of CEM.
  • Figure 3: Discrimination measures performances for the adult data: qq-plots comparing the distributions of the discrimination scores $D$ and $\delta$ for (i) the discrimination free data against the original data according to strategies (a)--(d) described in Sect. \ref{['sec:discadd']} (first two rows from the top) and (ii) the data where discrimination has been introduced against the original data, by setting $p_1=p_2=5$ and $p_1=p_2=10$ (bottom row).
  • Figure 4: Iterated CEM vs KNN (adult data): for $M=20$ simulated scenarios and threshold $q_D\in \{0.05, 0.1, 0.15, 0.2, 0.25\}$ the graphs show (i) the ratio of CEM and KNN correct prediction ratio (CPR), (ii) the ratio of CEM and KNN true positive ratio (TPR), and (iii) the ratio of CEM and KNN false negative ratio (FNR) (from top to bottom); the percentage of $S=1$ (respectively, $S=0$) observations whose outcome is changed to $Y=0$ (respectively, $Y=1$) is denoted by $v_1$ (respectively, $v_2$).
  • Figure 5: Discrimination measures performances for the COMPAS data: qq-plots comparing the distributions of the discrimination scores $D$ and $\delta$ for (i) the discrimination free data against the original data according to strategies (a)--(d) described in Sect. \ref{['sec:discadd']} (first two rows from the top) and (ii) the data where discrimination has been introduced against the original data, by setting $p_1=p_2=5$ and $p_1=p_2=10$ (bottom row).
  • ...and 6 more figures