Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BiMi Sheets: Infosheets for bias mitigation methods

MaryBeth Defrance, Guillaume Bied, Maarten Buyl, Jefrey Lijffijt, Tijl De Bie

TL;DR

This paper addresses the fragmentation and context-sensitivity of bias mitigation methods by introducing BiMi Sheets, a portable, uniform documentation framework. BiMi Sheets provide a six-section template to systematically capture a method's description, pipeline location, fairness formalization, implementation constraints, and use cases, accompanied by an open-platform hosting 24 ready-made sheets. The approach aims to improve comparability, benchmarking, and adoption by practitioners while enabling a structured database of bias mitigation methods. The authors acknowledge that documentation alone cannot solve the portability trap, but argue that standardized, transparent documentation is a crucial step toward more effective and context-aware bias mitigation in AI systems.

Abstract

Over the past 15 years, hundreds of bias mitigation methods have been proposed in the pursuit of fairness in machine learning (ML). However, algorithmic biases are domain-, task-, and model-specific, leading to a `portability trap': bias mitigation solutions in one context may not be appropriate in another. Thus, a myriad of design choices have to be made when creating a bias mitigation method, such as the formalization of fairness it pursues, and where and how it intervenes in the ML pipeline. This creates challenges in benchmarking and comparing the relative merits of different bias mitigation methods, and limits their uptake by practitioners. We propose BiMi Sheets as a portable, uniform guide to document the design choices of any bias mitigation method. This enables researchers and practitioners to quickly learn its main characteristics and to compare with their desiderata. Furthermore, the sheets' structure allow for the creation of a structured database of bias mitigation methods. In order to foster the sheets' adoption, we provide a platform for finding and creating BiMi Sheets at bimisheet.com.

BiMi Sheets: Infosheets for bias mitigation methods

TL;DR

This paper addresses the fragmentation and context-sensitivity of bias mitigation methods by introducing BiMi Sheets, a portable, uniform documentation framework. BiMi Sheets provide a six-section template to systematically capture a method's description, pipeline location, fairness formalization, implementation constraints, and use cases, accompanied by an open-platform hosting 24 ready-made sheets. The approach aims to improve comparability, benchmarking, and adoption by practitioners while enabling a structured database of bias mitigation methods. The authors acknowledge that documentation alone cannot solve the portability trap, but argue that standardized, transparent documentation is a crucial step toward more effective and context-aware bias mitigation in AI systems.

Abstract

Over the past 15 years, hundreds of bias mitigation methods have been proposed in the pursuit of fairness in machine learning (ML). However, algorithmic biases are domain-, task-, and model-specific, leading to a `portability trap': bias mitigation solutions in one context may not be appropriate in another. Thus, a myriad of design choices have to be made when creating a bias mitigation method, such as the formalization of fairness it pursues, and where and how it intervenes in the ML pipeline. This creates challenges in benchmarking and comparing the relative merits of different bias mitigation methods, and limits their uptake by practitioners. We propose BiMi Sheets as a portable, uniform guide to document the design choices of any bias mitigation method. This enables researchers and practitioners to quickly learn its main characteristics and to compare with their desiderata. Furthermore, the sheets' structure allow for the creation of a structured database of bias mitigation methods. In order to foster the sheets' adoption, we provide a platform for finding and creating BiMi Sheets at bimisheet.com.

Paper Structure

This paper contains 33 sections, 1 figure, 7 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Outline
  4. Related work
  5. Documentation for fairness in AI
  6. Datasheets for datasets Gebru_Morgenstern2021
  7. Model cards for models Mitchell_2019
  8. FactSheets by IBM Arnold2019
  9. Aequitas Saleiro_Kuester_Hinkson_London_Stevens_Anisfeld_Rodolfa_Ghani_2019
  10. BiMi Sheets in comparison
  11. Structuring for fairness in AI
  12. Documentation debt in bias mitigation
  13. Documentation for researchers
  14. Documentation for practitioners
  15. The current state of documentation
  16. ...and 18 more sections

Figures (1)

  • Figure 1: Bare-bones example of a BiMi Sheet. The symbol † denotes that the label in the tag is free to choose. The symbol * denotes that multiple tags for this property can be provided in a BiMiSheet.