Differential Adjusted Parity for Learning Fair Representations

TL;DR

The paper introduces Differential Adjusted Parity (DAP) loss, a differentiable, non-adversarial objective based on a differentiable variant of the adjusted parity metric, to jointly optimize task accuracy and invariance across sensitive domains via Soft Balanced Accuracy. It extends the adjusted parity metric to multiple domains and uses per-domain soft accuracy to form a differentiable objective that rewards high average accuracy while reducing cross-domain inconsistency. Empirically, DAP outperforms several adversarial debiasing methods on Adult and COMPAS datasets, achieving larger improvements in adjusted parity, equalized odds, and demographic parity metrics, while maintaining competitive task performance. While showing strong fairness gains, the method exhibits hyperparameter sensitivity, motivating future work on automatic tuning and dynamic adaptation of $\beta$ and $\Omega$ during training.

Abstract

The development of fair and unbiased machine learning models remains an ongoing objective for researchers in the field of artificial intelligence. We introduce the Differential Adjusted Parity (DAP) loss to produce unbiased informative representations. It utilises a differentiable variant of the adjusted parity metric to create a unified objective function. By combining downstream task classification accuracy and its inconsistency across sensitive feature domains, it provides a single tool to increase performance and mitigate bias. A key element in this approach is the use of soft balanced accuracies. In contrast to previous non-adversarial approaches, DAP does not suffer a degeneracy where the metric is satisfied by performing equally poorly across all sensitive domains. It outperforms several adversarial models on downstream task accuracy and fairness in our analysis. Specifically, it improves the demographic parity, equalized odds and sensitive feature accuracy by as much as 22.5\%, 44.1\% and 40.1\%, respectively, when compared to the best performing adversarial approaches on these metrics. Overall, the DAP loss and its associated metric can play a significant role in creating more fair machine learning models.

Figures (7)

• Figure 1: Comparing performance and fairness of all 4 models against DAP on Adult dataset. Top-left, top-right, bottom-left and bottom-right graph show how adjusted parity, equalised odds difference (EOD), gender classification accuracy and demographic parity difference (DPD) change with $\beta$. Higher adjusted parity and lower EOD, DPD and gender accuracy are favourable. DAP has higher adjusted parity and lower gender classification accuracy at all $\beta$. Lowest EOD and DPD are obtained by DAP at $\beta$=100.
• Figure 2: Effect of altering $\Omega$ and $\beta$ on adjusted parity (top-left), EOD (top-right), gender accuracy (bottom-left), and DPD (bottom-right) on Adult dataset. Higher adjusted parity and lower EOD, DPD and gender accuracy are favourable. Increasing $\beta$ lowers adjusted parity but improves all other metrics. Effect more pronounced at lower $\Omega$
• Figure 3: Comparing all 4 models performance and fairness against DAP on the COMPAS dataset. Top-left, top-right, bottom-left and bottom-right graph show how adjusted parity, EOD, race classification accuracy and DPD change with $\beta$. Higher adjusted parity and lower EOD, DPD and gender accuracy are favourable. DAP has lower gender classification accuracy at all $\beta$. Highest adjusted parity and lowest EOD and DPD are obtained by DAP
• Figure 4: Showing how the fairness metrics vary with changes in task accuracy from baseline values (Income Accuracy: 0.8294 for Adult, Recidivism Accuracy: 0.584 for COMPAS; EOD: 0.1429/0.310; DPD: 0.2759/0.173; Gender/Race Accuracy: 0.637/0.581) for each metric for Adult (left) and COMPAS (right). Changes were binned into 0.005 intervals, and averages for EOD, DPD, and gender/race accuracy were computed for each interval. Obtaining the largest negative change in EOD, DPD and sensitive feature accuracy for the least drop in task accuracy is favorable.
• Figure 5: Demonstrating the performance of DAP with multi-class sensitive features at $\Omega$=20. DPD and adjusted parity decrease and race classification accuracy approaches 0.33 with increasing $\beta$, as desired. EOD shows no significant trend.