Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mitigating Unfairness via Evolutionary Multi-objective Ensemble Learning

Qingquan Zhang, Jialin Liu, Zeqi Zhang, Junyi Wen, Bifei Mao, Xin Yao

TL;DR

Empirical results demonstrate that compared with the state-of-the-art approaches for mitigating unfairness, the proposed algorithm can provide decision makers with better tradeoffs among accuracy and multiple fairness metrics than other ensemble methods.

Abstract

In the literature of mitigating unfairness in machine learning, many fairness measures are designed to evaluate predictions of learning models and also utilised to guide the training of fair models. It has been theoretically and empirically shown that there exist conflicts and inconsistencies among accuracy and multiple fairness measures. Optimising one or several fairness measures may sacrifice or deteriorate other measures. Two key questions should be considered, how to simultaneously optimise accuracy and multiple fairness measures, and how to optimise all the considered fairness measures more effectively. In this paper, we view the mitigating unfairness problem as a multi-objective learning problem considering the conflicts among fairness measures. A multi-objective evolutionary learning framework is used to simultaneously optimise several metrics (including accuracy and multiple fairness measures) of machine learning models. Then, ensembles are constructed based on the learning models in order to automatically balance different metrics. Empirical results on eight well-known datasets demonstrate that compared with the state-of-the-art approaches for mitigating unfairness, our proposed algorithm can provide decision-makers with better tradeoffs among accuracy and multiple fairness metrics. Furthermore, the high-quality models generated by the framework can be used to construct an ensemble to automatically achieve a better tradeoff among all the considered fairness metrics than other ensemble methods. Our code is publicly available at https://github.com/qingquan63/FairEMOL

Mitigating Unfairness via Evolutionary Multi-objective Ensemble Learning

TL;DR

Empirical results demonstrate that compared with the state-of-the-art approaches for mitigating unfairness, the proposed algorithm can provide decision makers with better tradeoffs among accuracy and multiple fairness metrics than other ensemble methods.

Abstract

In the literature of mitigating unfairness in machine learning, many fairness measures are designed to evaluate predictions of learning models and also utilised to guide the training of fair models. It has been theoretically and empirically shown that there exist conflicts and inconsistencies among accuracy and multiple fairness measures. Optimising one or several fairness measures may sacrifice or deteriorate other measures. Two key questions should be considered, how to simultaneously optimise accuracy and multiple fairness measures, and how to optimise all the considered fairness measures more effectively. In this paper, we view the mitigating unfairness problem as a multi-objective learning problem considering the conflicts among fairness measures. A multi-objective evolutionary learning framework is used to simultaneously optimise several metrics (including accuracy and multiple fairness measures) of machine learning models. Then, ensembles are constructed based on the learning models in order to automatically balance different metrics. Empirical results on eight well-known datasets demonstrate that compared with the state-of-the-art approaches for mitigating unfairness, our proposed algorithm can provide decision-makers with better tradeoffs among accuracy and multiple fairness metrics. Furthermore, the high-quality models generated by the framework can be used to construct an ensemble to automatically achieve a better tradeoff among all the considered fairness metrics than other ensemble methods. Our code is publicly available at https://github.com/qingquan63/FairEMOL
Paper Structure (25 sections, 5 equations, 7 figures, 9 tables, 3 algorithms)

This paper contains 25 sections, 5 equations, 7 figures, 9 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Measuring Fairness in Machine Learning
  4. Mitigating Unfairness in Machine Learning
  5. Multi-objective Evolutionary Learning and Ensemble Learning for Mitigating Unfairness
  6. Multi-objective Learning Framework for Fairer ML
  7. Multi-objective Ensemble Learning Framework for Fairer ML
  8. Proposed Algorithms based on Our Framework
  9. Model set
  10. Evaluation criteria
  11. Multi-objective optimiser
  12. Multi-objective ensemble selection strategy
  13. Experimental Studies
  14. An Overview of the Two Studies
  15. Answering $\mathcal{Q}1$ and $\mathcal{Q}2$
  16. ...and 10 more sections

Figures (7)

  • Figure 1: Evaluated $F_{EI}$, $F_{EG}$ and $F_{EIG}$ values (left to right) on the test set. Different colours indicate solutions at different generations. Green stars highlight the non-dominated solutions in the final generation.
  • Figure 2: HV values averaged over 30 trials considering their corresponding objectives on the set set. x-axis: generation number; y-axis: HV value.
  • Figure 3: Illustrative examples: (i) non-dominated solutions of the models obtained in the whole evolution process of $F_{EIG}$ in 30 trials (black points). (ii) non-dominated solutions of the models in the last generation of $F_{EIG}$ in one arbitrary trial (green triangles).
  • Figure 4: Illustrative examples: (i) the models obtained by different Multi-FRs in all 30 trials (colourful points); (ii) non-dominated solutions of the models in the last generation of $F_{EIG}$ in one arbitrary trial (black points).
  • Figure 5: HV values of models in Metric Sets I-III, respectively, averaged over 30 trials on 15 datasets by applying our algorithm to directly optimise the Metric Set I. x-axis: generation number; y-axis: HV value.
  • ...and 2 more figures