Fairness-aware Multiobjective Evolutionary Learning

TL;DR

The proposed framework achieves outstanding performance compared to the state-of-the-art approaches for mitigating unfairness in terms of accuracy as well as 25 fairness measures although only a few of them were dynamically selected and used as optimization objectives.

Abstract

Multiobjective evolutionary learning (MOEL) has demonstrated its advantages of training fairer machine learning models considering a predefined set of conflicting objectives, including accuracy and different fairness measures. Recent works propose to construct a representative subset of fairness measures as optimisation objectives of MOEL throughout model training. However, the determination of a representative measure set relies on dataset, prior knowledge and requires substantial computational costs. What's more, those representative measures may differ across different model training processes. Instead of using a static predefined set determined before model training, this paper proposes to dynamically and adaptively determine a representative measure set online during model training. The dynamically determined representative set is then used as optimising objectives of the MOEL framework and can vary with time. Extensive experimental results on 12 well-known benchmark datasets demonstrate that our proposed framework achieves outstanding performance compared to state-of-the-art approaches for mitigating unfairness in terms of accuracy as well as 25 fairness measures although only a few of them were dynamically selected and used as optimisation objectives. The results indicate the importance of setting optimisation objectives dynamically during training.

Figures (9)

• Figure 1: Flow of our framework, where a fairness-aware strategy is used to dynamically select a representative subset (solid circles) to be optimised using MOEL at each generation to improve all the measures (all circles). Each generation involves mating selection, reproduction and survival selection, encompassing the loop outlined in lines 6-13 of Algorithm 1, which will be detailed in Section III.A.
• Figure 2: HV curves along with generations averaged over 50 trials considering accuracy and $f_1$--$f_{25}$
• Figure 3: Visualisation of the fairness awareness process of $FaMOEL$. The first and second columns depict the evolution of the representative objective subset to be optimised at each generation. Each light-colored block represents the selection of an objective (corresponding to its respective column) for optimisation at the corresponding generation (corresponding to the row). The third column displays the average frequency of selecting each objective along with 100 generations over 50 trials.
• Figure 4: Heatmap illustrating the correlation among accuracy and 25 fairness measures at generations 1, 50 and 100, respectively, in dealing with Drug consumption.
• Figure 5: Number of measures selected as objectives at each generation within two arbitrary trials on LSAT dataset.