Causal foundations of bias, disparity and fairness

TL;DR

This paper defines bias as a direct causal effect that is unjustified and disparity as a direct or indirect causal effect that includes a bias, all within structural causal models. It argues that many traditional AI fairness criteria (independence, separation, sufficiency) can misclassify fairness by ignoring underlying causal structure, and it advocates counterfactual and path-specific causal reasoning to assess fairness. The authors discuss methodological challenges such as conditioning on colliders and unobserved confounders, and illustrate the framework with gender bias in science and racial bias in police shootings. They further discuss policy interventions and emphasize that addressing the root unjustified direct effects is crucial for effective and ethical remedies. Overall, the work provides a rigorous causal foundation for distinguishing bias from disparity and for guiding policy and AI fairness research.

Abstract

The study of biases, such as gender or racial biases, is an important topic in the social and behavioural sciences. However, the literature does not always clearly define the concept. Definitions of bias are often ambiguous or not provided at all. To study biases in a precise manner, it is important to have a well-defined concept of bias. We propose to define bias as a direct causal effect that is unjustified. We propose to define the closely related concept of disparity as a direct or indirect causal effect that includes a bias. Our proposed definitions can be used to study biases and disparities in a more rigorous and systematic way. We compare our definitions of bias and disparity with various criteria of fairness introduced in the artificial intelligence literature. In addition, we discuss how our definitions relate to discrimination. We illustrate our definitions of bias and disparity in two case studies, focusing on gender bias in science and racial bias in police shootings. Our proposed definitions aim to contribute to a better appreciation of the causal intricacies of studies of biases and disparities. We hope that this will also promote an improved understanding of the policy implications of such studies.

Figures (8)

• Figure 1: Illustration of our terminology.
• Figure 2: Simple hypothetical example illustrating our definitions of bias and disparity. Gender has a direct causal effect on productivity. This effect is considered unjustified and is therefore coloured red. We say there is a gender bias in productivity. Productivity and impact both have direct causal effects on faculty position. These effects are considered justified. The unjustified effect of gender on productivity affects faculty position indirectly. Therefore, both productivity and faculty position are considered unfair outcomes of gender and, hence, coloured red. We say there is a gender disparity in productivity and faculty position.
• Figure 3: Example of a simple DAG. On a path between nodes $X$ and $Y$, node $Z$ is a confounder (hence, open), and node $Q$ is a collider (hence, closed).
• Figure 4: Illustration showing when a node $Z$ on an undirected path is open (coloured green) or closed (coloured red). Conditioning on a variable 'flips' a node from open to closed or vice versa.
• Figure 5: Challenges in identifying biases and disparities. The variables $U$ and $Q$ are assumed to be unobserved or unobservable, making it impossible to control for them.