FairPlay: A Collaborative Approach to Mitigate Bias in Datasets for Improved AI Fairness
Tina Behzad, Mithilesh Kumar Singh, Anthony J. Ripa, Klaus Mueller
TL;DR
FairPlay introduces a collaborative, multi-user extension of the D-BIAS framework that enables stakeholders to negotiate and adjust causal graphs for debiasing datasets in a pre-processing setting. By converting bias mitigation into a structured, game-based negotiation over edge weights in a causal network, FairPlay achieves consensus among diverse perspectives, demonstrated across four user studies. The approach yields debiased datasets that improve fairness metrics (e.g., individual fairness and parity) at the cost of some accuracy, and is complemented by rich visualizations, metrics, and usability analysis. This work highlights the practical value of human-centered, consensus-driven bias mitigation for AI systems and outlines directions for broader applicability and enhancement.
Abstract
The issue of fairness in decision-making is a critical one, especially given the variety of stakeholder demands for differing and mutually incompatible versions of fairness. Adopting a strategic interaction of perspectives provides an alternative to enforcing a singular standard of fairness. We present a web-based software application, FairPlay, that enables multiple stakeholders to debias datasets collaboratively. With FairPlay, users can negotiate and arrive at a mutually acceptable outcome without a universally agreed-upon theory of fairness. In the absence of such a tool, reaching a consensus would be highly challenging due to the lack of a systematic negotiation process and the inability to modify and observe changes. We have conducted user studies that demonstrate the success of FairPlay, as users could reach a consensus within about five rounds of gameplay, illustrating the application's potential for enhancing fairness in AI systems.