Equalised Odds is not Equal Individual Odds: Post-processing for Group and Individual Fairness

TL;DR

This work confronts the incompatibility of group fairness (equalised odds) and individual fairness under fixed randomisation, showing that threshold-based post-processing can create discontinuities and unequal access to favorable odds. It introduces continuous, monotone probability curves between group thresholds, constrained by a Lipschitz constant, and derives them via linear systems to satisfy both equalised odds and a novel equalised individual odds criterion. Through CreditRisk and COMPAS case studies, the authors demonstrate that these curves preserve predictive accuracy while improving individual fairness and fairness exposure across sub-populations. The approach enhances transparency and user incentives to improve scores, offering a practical, implementable post-processing solution for black-box scoring systems with high-stakes implications.

Abstract

Group fairness is achieved by equalising prediction distributions between protected sub-populations; individual fairness requires treating similar individuals alike. These two objectives, however, are incompatible when a scoring model is calibrated through discontinuous probability functions, where individuals can be randomly assigned an outcome determined by a fixed probability. This procedure may provide two similar individuals from the same protected group with classification odds that are disparately different -- a clear violation of individual fairness. Assigning unique odds to each protected sub-population may also prevent members of one sub-population from ever receiving equal chances of a positive outcome to another, which we argue is another type of unfairness called individual odds. We reconcile all this by constructing continuous probability functions between group thresholds that are constrained by their Lipschitz constant. Our solution preserves the model's predictive power, individual fairness and robustness while ensuring group fairness.

Figures (9)

• Figure 1: Two-threshold fixed randomisation NIPS2016_9d268236 applied to probabilities (y-axis) output by a loan repayment classifier built upon credit scores (x-axis). It satisfies equalised odds for the binary protected attribute race (black and white) by using it to assign approval probabilities, but results in discontinuities that violate individual fairness and create a gap between group-specific individual odds.
• Figure 2: ROC curves for the CreditRisk data set. The solution space for each (protected) group is given by all the points on their respective ROC curve when a single threshold is used. If we rely on multiple thresholds and randomisation, however, we expand the solution space to all the points on and below an ROC curve -- represented for each group as the coloured area. A fair solution, according to equalised odds, is any set of thresholds and probabilities such that each group achieves equal true and false positives.
• Figure 3: Two-threshold preferential randomisation with smoothness constraints applied to probabilities (y-axis) output by a loan repayment classifier built upon credit scores (x-axis). It satisfies equalised odds for the protected attribute race (black and white) by using it to assign approval probabilities. This solution has no discontinuities -- satisfying individual odds (Definition \ref{['individualodds']}, see $r$ and $r^\prime$ for an example) and being $L_\mathcal{R}$ Lipschitz-continuous (Equation \ref{['lip']}) -- and offers predictive performance marginally better than the fixed randomisation method shown in Figure \ref{['multi-threshold']}.
• Figure 4: Probability curves corresponding to the results reported in Table \ref{['results']}. All solutions have comparable accuracy and satisfy equalised odds but yield a different Lipschitz constant $L_\mathcal{R}$. The Hispanic group is omitted as it uses a single threshold $t_{y,a}=30$ (refer to Table \ref{['params']} given in Appendix \ref{['ap:thresh']}).
• Figure 5: ROC curves for the COMPAS data set. The coloured regions indicate areas accessible to each group. (Refer to Figure \ref{['ROC']} for more details.)

Theorems & Definitions (3)

• Definition 2.1: Equalised Odds
• Definition 3.1: Classification Odds Distance
• Definition 3.2: Equalised Individual Odds