On the Origins of Sampling Bias: Implications on Fairness Measurement and Mitigation

TL;DR

This work formalizes sampling biases in fairness evaluation by introducing sample size bias ($SSB$) and underrepresentation bias ($URB$) and empirically investigates their impact across benchmarks, metrics, and classifiers. It demonstrates that metrics combining sensitivity and specificity, such as $AUC$ and $ZOL$, tend to be more robust to these biases than traditional metrics like $FPR$ or $EO$, especially under small or imbalanced data. The study also shows that bias mitigation techniques are less effective when data are scarce or unevenly represented, and that data augmentation can either amplify or reduce discrimination depending on the strategy and task. The findings yield actionable practitioner recommendations, including preferred metrics for biased or small datasets and cautions on data augmentation, to support fairer ML deployment.

Abstract

Accurately measuring discrimination is crucial to faithfully assessing fairness of trained machine learning (ML) models. Any bias in measuring discrimination leads to either amplification or underestimation of the existing disparity. Several sources of bias exist and it is assumed that bias resulting from machine learning is born equally by different groups (e.g. females vs males, whites vs blacks, etc.). If, however, bias is born differently by different groups, it may exacerbate discrimination against specific sub-populations. Sampling bias, in particular, is inconsistently used in the literature to describe bias due to the sampling procedure. In this paper, we attempt to disambiguate this term by introducing clearly defined variants of sampling bias, namely, sample size bias (SSB) and underrepresentation bias (URB). Through an extensive set of experiments on benchmark datasets and using mainstream learning algorithms, we expose relevant observations in several model training scenarios. The observations are finally framed as actionable recommendations for practitioners.

Figures (33)

• Figure 1: Magnitude of sample size bias (SSB) for increasing size of the training data.
• Figure 2: Behavior of discrimination metrics (along with protected group values) while increasing the training data size sampled from very discriminative population (postive outcome is 0.9 for privileged group and 0.1 for unprivileged group).
• Figure 3: Importance of the sensitive attribute (Sex in Adult dataset) while increasing the data samples used for training.
• Figure 4: Magnitude of SSB with varying level of imbalance and increasing size of the training data.
• Figure 5: Magnitude of discrimination on increasingly imbalaned training data while increasing the training data size.

Theorems & Definitions (2)

• Definition 3.1
• Definition 3.2