No evaluation without fair representation : Impact of label and selection bias on the evaluation, performance and mitigation of classification models

TL;DR

This work empirically analyze the effect of label bias and several subtypes of selection bias on the evaluation of classification models, on their performance, and on the effectiveness of bias mitigation methods, and introduces a biasing and evaluation framework that allows to model fair worlds and their biased counterparts through the introduction of controlled bias in real-life datasets with low discrimination.

Abstract

Bias can be introduced in diverse ways in machine learning datasets, for example via selection or label bias. Although these bias types in themselves have an influence on important aspects of fair machine learning, their different impact has been understudied. In this work, we empirically analyze the effect of label bias and several subtypes of selection bias on the evaluation of classification models, on their performance, and on the effectiveness of bias mitigation methods. We also introduce a biasing and evaluation framework that allows to model fair worlds and their biased counterparts through the introduction of controlled bias in real-life datasets with low discrimination. Using our framework, we empirically analyze the impact of each bias type independently, while obtaining a more representative evaluation of models and mitigation methods than with the traditional use of a subset of biased data as test set. Our results highlight different factors that influence how impactful bias is on model performance. They also show an absence of trade-off between fairness and accuracy, and between individual and group fairness, when models are evaluated on a test set that does not exhibit unwanted bias. They furthermore indicate that the performance of bias mitigation methods is influenced by the type of bias present in the data. Our findings call for future work to develop more accurate evaluations of prediction models and fairness interventions, but also to better understand other types of bias, more complex scenarios involving the combination of different bias types, and other factors that impact the efficiency of the mitigation methods, such as dataset characteristics.

Figures (34)

• Figure 1: Fair World Framework. The fair world represents an ideal situation in which the desired fairness criteria hold. The available observable data is a distorted representation of that world, typically obtained through an unknown biasing process. Such data is traditionally used to both train prediction models and evaluate their performance.
• Figure 2: Comparison of metric results evaluated on a biased test set (Biased evaluation) or a fair test set (Fair evaluation) for values in $(-1,1)$. Points that fall outside of the x=y diagonal indicate that the metric measurement is skewed by the bias present in the unfair test set. Are included the results for 4 datasets with bias intensities ranging from 0 to 0.9, 3 fairness-agnostic training algorithms, 8 bias mitigation procedures, and unmitigated models.
• Figure 3: Biasing and evaluation framework. We assume that a given dataset with high fairness level, the fair data, accurately represents the fair world. We introduce a specific bias type in this fair data to obtain datasets with known bias. This biased data can be used to train prediction models with or without bias mitigation. The performance and fairness of these models can then be evaluated using the fair data. They can also be compared to baseline models trained on that fair data.
• Figure 4: Experiment pipeline representing data biasing, model training (with or without bias mitigation), and evaluation of models on fair data. Thin dashed arrows indicate where each type of data is used. Bold dashed arrows indicate an optional path. We do not cumulate the use of pre- and pot-processing methods.
• Figure 5: Fair evaluation