Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MESD: Detecting and Mitigating Procedural Bias in Intersectional Groups

Gideon Popoola, John Sheppard

Abstract

Research about bias in machine learning has mostly focused on outcome-oriented fairness metrics (e.g., equalized odds) and on a single protected category. Although these approaches offer great insight into bias in ML, they provide limited insight into model procedure bias. To address this gap, we proposed multi-category explanation stability disparity (MESD), an intersectional, procedurally oriented metric that measures the disparity in the quality of explanations across intersectional subgroups in multiple protected categories. MESD serves as a complementary metric to outcome-oriented metrics, providing detailed insight into the procedure of a model. To further extend the scope of the holistic selection model, we also propose a multi-objective optimization framework, UEF (Utility-Explanation-Fairness), that jointly optimizes three objectives. Experimental results across multiple datasets show that UEF effectively balances objectives. Also, the results show that MESD can effectively capture the explanation difference between intersectional groups. This research addresses an important gap by examining explainability with respect to fairness across multiple protected categories.

MESD: Detecting and Mitigating Procedural Bias in Intersectional Groups

Abstract

Research about bias in machine learning has mostly focused on outcome-oriented fairness metrics (e.g., equalized odds) and on a single protected category. Although these approaches offer great insight into bias in ML, they provide limited insight into model procedure bias. To address this gap, we proposed multi-category explanation stability disparity (MESD), an intersectional, procedurally oriented metric that measures the disparity in the quality of explanations across intersectional subgroups in multiple protected categories. MESD serves as a complementary metric to outcome-oriented metrics, providing detailed insight into the procedure of a model. To further extend the scope of the holistic selection model, we also propose a multi-objective optimization framework, UEF (Utility-Explanation-Fairness), that jointly optimizes three objectives. Experimental results across multiple datasets show that UEF effectively balances objectives. Also, the results show that MESD can effectively capture the explanation difference between intersectional groups. This research addresses an important gap by examining explainability with respect to fairness across multiple protected categories.
Paper Structure (17 sections, 23 equations, 2 figures, 2 tables)

This paper contains 17 sections, 23 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The MESD Method
  4. Intersectional Subgroups
  5. Aggregation of Explainers
  6. Perturbation
  7. Stability Evaluation
  8. Label-Aware Group Aggregation
  9. CVaR-Weighted Group Disparity (MESD)
  10. Optimizing Fairness and Explainability
  11. Experimental Evaluation
  12. Experimental Settings
  13. Performance Comparison
  14. Comparison of each algorithm on the objectives
  15. Evaluating CVaR on MESD
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Pareto Fronts of each algorithm on the Adult Income dataset
  • Figure 2: MESD variants from each algorithm on the Recidivism Dataset