Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diversity Drives Fairness: Ensemble of Higher Order Mutants for Intersectional Fairness of Machine Learning Software

Zhenpeng Chen, Xinyue Li, Jie M. Zhang, Federica Sarro, Yang Liu

TL;DR

FairHOME tackles the challenge of intersectional fairness in ML by performing higher-order mutation of input attributes during inference to create diverse subgroups, and then ensembling predictions from the same model. It avoids retraining, enabling applicability to deployed systems, and uses majority voting as a simple, effective default ensemble. In a large empirical study across 24 tasks and 7 baselines, FairHOME achieves an average 47.5% improvement in intersectional fairness with only a small ML-performance cost (0.1%–2.7%), and demonstrates the strongest fairness-performance trade-off using the Fairea benchmark. The approach also shows benefits for single-attribute group fairness and remains robust across ensemble strategies and mutation settings, making it a practical, scalable bias-mitigation technique for real-world ML software.

Abstract

Intersectional fairness is a critical requirement for Machine Learning (ML) software, demanding fairness across subgroups defined by multiple protected attributes. This paper introduces FairHOME, a novel ensemble approach using higher order mutation of inputs to enhance intersectional fairness of ML software during the inference phase. Inspired by social science theories highlighting the benefits of diversity, FairHOME generates mutants representing diverse subgroups for each input instance, thus broadening the array of perspectives to foster a fairer decision-making process. Unlike conventional ensemble methods that combine predictions made by different models, FairHOME combines predictions for the original input and its mutants, all generated by the same ML model, to reach a final decision. Notably, FairHOME is even applicable to deployed ML software as it bypasses the need for training new models. We extensively evaluate FairHOME against seven state-of-the-art fairness improvement methods across 24 decision-making tasks using widely adopted metrics. FairHOME consistently outperforms existing methods across all metrics considered. On average, it enhances intersectional fairness by 47.5%, surpassing the currently best-performing method by 9.6 percentage points.

Diversity Drives Fairness: Ensemble of Higher Order Mutants for Intersectional Fairness of Machine Learning Software

TL;DR

FairHOME tackles the challenge of intersectional fairness in ML by performing higher-order mutation of input attributes during inference to create diverse subgroups, and then ensembling predictions from the same model. It avoids retraining, enabling applicability to deployed systems, and uses majority voting as a simple, effective default ensemble. In a large empirical study across 24 tasks and 7 baselines, FairHOME achieves an average 47.5% improvement in intersectional fairness with only a small ML-performance cost (0.1%–2.7%), and demonstrates the strongest fairness-performance trade-off using the Fairea benchmark. The approach also shows benefits for single-attribute group fairness and remains robust across ensemble strategies and mutation settings, making it a practical, scalable bias-mitigation technique for real-world ML software.

Abstract

Intersectional fairness is a critical requirement for Machine Learning (ML) software, demanding fairness across subgroups defined by multiple protected attributes. This paper introduces FairHOME, a novel ensemble approach using higher order mutation of inputs to enhance intersectional fairness of ML software during the inference phase. Inspired by social science theories highlighting the benefits of diversity, FairHOME generates mutants representing diverse subgroups for each input instance, thus broadening the array of perspectives to foster a fairer decision-making process. Unlike conventional ensemble methods that combine predictions made by different models, FairHOME combines predictions for the original input and its mutants, all generated by the same ML model, to reach a final decision. Notably, FairHOME is even applicable to deployed ML software as it bypasses the need for training new models. We extensively evaluate FairHOME against seven state-of-the-art fairness improvement methods across 24 decision-making tasks using widely adopted metrics. FairHOME consistently outperforms existing methods across all metrics considered. On average, it enhances intersectional fairness by 47.5%, surpassing the currently best-performing method by 9.6 percentage points.

Paper Structure

This paper contains 33 sections, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Fairness in ML Software
  4. Related Work
  5. Our approach
  6. FairHOME: In a Nutshell
  7. Input Generation
  8. Output Ensemble
  9. Evaluation Setup
  10. Research Questions (RQs)
  11. Experimental Methodology
  12. Step 1. Design of bias mitigation tasks
  13. Step 2. Selection of baseline methods
  14. Step 3. Analysis of FairHOME's effect
  15. Step 4. Comparison of intersectional fairness
  16. ...and 18 more sections

Figures (2)

  • Figure 1: Overview of FairHOME.
  • Figure 2: (RQ3) Effectiveness level distributions of FairHOME and existing methods in fairness-performance trade-off. Overall, FairHOME achieves the best trade-off, with 87.7% of mitigation cases falling in win-win or good trade-off.