Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Measuring and Mitigating Bias for Tabular Datasets with Multiple Protected Attributes

Manh Khoi Duong, Stefan Conrad

TL;DR

This paper proposes and presents measures and mitigation strategies for discrimination in tabular datasets that contain multiple protected attributes, such as nationality, age, and sex, and shows that de-biasing datasets with multiple protected attributes is possible.

Abstract

Motivated by the recital (67) of the current corrigendum of the AI Act in the European Union, we propose and present measures and mitigation strategies for discrimination in tabular datasets. We specifically focus on datasets that contain multiple protected attributes, such as nationality, age, and sex. This makes measuring and mitigating bias more challenging, as many existing methods are designed for a single protected attribute. This paper comes with a twofold contribution: Firstly, new discrimination measures are introduced. These measures are categorized in our framework along with existing ones, guiding researchers and practitioners in choosing the right measure to assess the fairness of the underlying dataset. Secondly, a novel application of an existing bias mitigation method, FairDo, is presented. We show that this strategy can mitigate any type of discrimination, including intersectional discrimination, by transforming the dataset. By conducting experiments on real-world datasets (Adult, Bank, COMPAS), we demonstrate that de-biasing datasets with multiple protected attributes is possible. All transformed datasets show a reduction in discrimination, on average by 28%. Further, these datasets do not compromise any of the tested machine learning models' performances significantly compared to the original datasets. Conclusively, this study demonstrates the effectiveness of the mitigation strategy used and contributes to the ongoing discussion on the implementation of the European Union's AI Act.

Measuring and Mitigating Bias for Tabular Datasets with Multiple Protected Attributes

TL;DR

This paper proposes and presents measures and mitigation strategies for discrimination in tabular datasets that contain multiple protected attributes, such as nationality, age, and sex, and shows that de-biasing datasets with multiple protected attributes is possible.

Abstract

Motivated by the recital (67) of the current corrigendum of the AI Act in the European Union, we propose and present measures and mitigation strategies for discrimination in tabular datasets. We specifically focus on datasets that contain multiple protected attributes, such as nationality, age, and sex. This makes measuring and mitigating bias more challenging, as many existing methods are designed for a single protected attribute. This paper comes with a twofold contribution: Firstly, new discrimination measures are introduced. These measures are categorized in our framework along with existing ones, guiding researchers and practitioners in choosing the right measure to assess the fairness of the underlying dataset. Secondly, a novel application of an existing bias mitigation method, FairDo, is presented. We show that this strategy can mitigate any type of discrimination, including intersectional discrimination, by transforming the dataset. By conducting experiments on real-world datasets (Adult, Bank, COMPAS), we demonstrate that de-biasing datasets with multiple protected attributes is possible. All transformed datasets show a reduction in discrimination, on average by 28%. Further, these datasets do not compromise any of the tested machine learning models' performances significantly compared to the original datasets. Conclusively, this study demonstrates the effectiveness of the mitigation strategy used and contributes to the ongoing discussion on the implementation of the European Union's AI Act.
Paper Structure (24 sections, 16 equations, 2 figures, 3 tables)

This paper contains 24 sections, 16 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Measuring Discrimination for Multiple Attributes
  4. Single Protected Attribute $(|\mathcal{Z}| = 1)$
  5. No Groups ($|g| = 0$)
  6. Single Group ($|g| = 1$)
  7. Binary Groups ($|g| = 2$)
  8. Non-binary Groups ($|g| > 2$)
  9. Multiple Protected Attributes $(|\mathcal{Z}| > 1)$
  10. Intersectional Discrimination
  11. Non-intersectional Discrimination
  12. Example
  13. Experiments
  14. Experimental Setup
  15. Datasets and Pre-processing
  16. ...and 9 more sections

Figures (2)

  • Figure 1: Stick figures can be differentiated by their color and shape. In intersectional discrimination, attributes are intersected, which leads to new subgroups. In non-intersectional, each attribute is treated independently, i.e., colors and shapes are not intersecting in this case.
  • Figure 2: Results on the test set. The x-axis represents the discrimination values (legend indicates used measure) and the y-axis represents the classifiers' performances. We compare the pre-processed (fair) data with the original data. The points/stars represent averages, and the error bars display the standard deviations of the AUROC and discrimination values over 10 trials.

Theorems & Definitions (8)

  • Definition 2.1: Treatment
  • Definition 2.2: Fairness Criteria
  • Example 2.1: Statistical Parity calders2009
  • Definition 2.3: Disparity
  • Definition 2.4: Discrimination
  • Definition 3.1: Subgroup pmlr-v80-kearns18a
  • Definition 3.2: Subgroup Treatment
  • Definition 3.3: Subgroup Disparity