Analyzing Fairness of Classification Machine Learning Model with Structured Dataset

TL;DR

The paper addresses fairness in ML classification on structured data and the risk of biased predictions perpetuating inequities. It adopts an empirical approach comparing Fairlearn, AIF360, and What-If Tool using the Adult Income dataset from the UCI Repository, with preprocessing, in-processing, and post-processing mitigation strategies. Key contributions include a comparative assessment of the libraries' strengths and limitations, and demonstrated mitigation strategies achieving substantial reductions in bias while maintaining or improving accuracy. The findings offer actionable guidance for practitioners to integrate fairness tools into real-world ML workflows, advancing the development of more equitable AI systems.

Abstract

Machine learning (ML) algorithms have become integral to decision making in various domains, including healthcare, finance, education, and law enforcement. However, concerns about fairness and bias in these systems pose significant ethical and social challenges. This study investigates the fairness of ML models applied to structured datasets in classification tasks, highlighting the potential for biased predictions to perpetuate systemic inequalities. A publicly available dataset from Kaggle was selected for analysis, offering a realistic scenario for evaluating fairness in machine learning workflows. To assess and mitigate biases, three prominent fairness libraries; Fairlearn by Microsoft, AIF360 by IBM, and the What If Tool by Google were employed. These libraries provide robust frameworks for analyzing fairness, offering tools to evaluate metrics, visualize results, and implement bias mitigation strategies. The research aims to assess the extent of bias in the ML models, compare the effectiveness of these libraries, and derive actionable insights for practitioners. The findings reveal that each library has unique strengths and limitations in fairness evaluation and mitigation. By systematically comparing their capabilities, this study contributes to the growing field of ML fairness by providing practical guidance for integrating fairness tools into real world applications. These insights are intended to support the development of more equitable machine learning systems.