Mutatis Mutandis: Revisiting the Comparator in Discrimination Testing

TL;DR

This paper reframes discrimination testing around the comparator by treating its derivation as a causal modeling problem and introducing a new MM (mutatis mutandis) comparator that accounts for how the protected attribute affects non-protected attributes. It contrasts CP (ceteris paribus) with MM, arguing that MM better embodies substantive equality by adjusting for downstream effects of the protected attribute, and demonstrates this via a Law School Admissions experiment that shows MM-based testing (CST) detects more prima facie discrimination than CP-based testing (ST). The authors formalize MM using protected-aware distances and counterfactual representations, linking MM to fair representation learning and SCM-based counterfactual generation, and discuss broader implications for discrimination discovery and algorithmic fairness tools. They call for incorporating MM-style comparators into discrimination testing pipelines to better reflect normative goals and provide a practical path for ML methods to implement these richer comparisons, while acknowledging limitations and avenues for future work in causal modeling and legal interpretation.

Abstract

Testing for individual discrimination involves deriving a profile, the comparator, similar to the one making the discrimination claim, the complainant, based on a protected attribute, such as race or gender, and comparing their decision outcomes. The complainant-comparator pair is central to discrimination testing. Most discrimination testing tools rely on this pair to establish evidence for discrimination. In this work we revisit the role of the comparator in discrimination testing. We first argue for the inherent causal modeling nature of deriving the comparator. We then introduce a two-kinds classification for the comparator: the ceteris paribus, or``with all else equal,'' (CP) comparator and the mutatis mutandis, or ``with the appropriate adjustments being made,'' (MM) comparator. The CP comparator is the standard comparator, representing an idealized comparison for establishing discrimination as it aims for a complainant-comparator pair that only differs on membership to the protected attribute. As an alternative to it, we define the MM comparator, which requires that the comparator represents the``what would have been of'' the complainant without the effects of the protected attribute on the non-protected attributes. Under the MM comparator, the complainant-comparator pair can be dissimilar in terms of the non-protected attributes, departing from an idealized comparison. Notably, the MM comparator is a more complex kind of comparator and its implementation offers an impactful venue for machine learning methods. We illustrate these two comparators and their impact on discrimination testing using a real-world example.

Figures (4)

• Figure 1: The auxiliary causal knowledge with corresponding SCM $\mathcal{M}$ and DAG $\mathcal{G}$ for Law School Admissions (Level 3) Kusner2017CF. Let $b_U$ and $b_L$ denote the intercepts and$\beta_1$, $\beta_2$, $\lambda_1$, $\lambda_2$ the regression weights. These parameters are estimated from the data using ordinary linear regression. We obtain, respectively, the estimates $\hat{b_U}=3.21$, $\hat{\beta_1}=-0.22$, $\hat{\lambda_1}=0.13$ and $\hat{b_L}=37.8$, $\hat{\beta_2}=-4.64$, $\hat{\lambda_2}=-0.61$.
• Figure 2: We show the distributions in the form of box-plots for the $LSAT$ and $UGPA$ for seven randomly chosen complainants based on the protected attribute gender. These complainants are found to be discriminated by both ST and CST for $k=100$. The so-called control (ctr) are the neighborhoods of similar female profiles to the complainant, which is the same for both tools. Similarly, the so-called test groups (tst-st for ST; tst-cf for CST) are the neighborhoods of similar male profiles to the comparator: i.e., the CP comparator for ST, and the MM comparator for CST. In both figures we observe a difference between the two kinds of test groups. As we would expect, it means that the test and control groups are more similar under ST than under CST. Both figures show the impact of the CP and MM comparators.
• Figure 3: The same analysis follows as in Figure \ref{['fig:gender_analysis']} but based on the protected attribute race, including similar insights and conclusions on the influence of the CP and MM comparators. In both figures, we observe a difference between the two kinds of test groups, highlighting again the impact of the CP and MM comparators.
• Figure :

Theorems & Definitions (3)

• definition 1
• definition 2
• definition 3