FairRF: Multi-Objective Search for Single and Intersectional Software Fairness

TL;DR

FairRF addresses the problem of bias mitigation in AI systems by framing it as a multi-objective optimization task over a Random Forest base classifier with data mutations. It jointly maximizes classification effectiveness ($Accuracy$) and minimizes fairness violations ($SPD$), producing a Pareto front that enables stakeholders to select solutions aligned with their priorities. The approach is extensively evaluated against a broad set of baselines and SOTA methods across multiple datasets, demonstrating strong improvements in fairness while preserving, and sometimes enhancing, overall performance, including robust handling of intersectional bias. The work provides a replication package and argues for the practical utility of MOEA-based, user-guided fairness enhancement in software systems, with clear paths for future extension to broader tasks and richer mutation strategies.

Abstract

Background: The wide adoption of AI- and ML-based systems in sensitive domains raises severe concerns about their fairness. Many methods have been proposed in the literature to enhance software fairness. However, the majority behave as a black-box, not allowing stakeholders to prioritise fairness or effectiveness (i.e., prediction correctness) based on their needs. Aims: In this paper, we introduce FairRF, a novel approach based on multi-objective evolutionary search to optimise fairness and effectiveness in classification tasks. FairRF uses a Random Forest (RF) model as a base classifier and searches for the best hyperparameter configurations and data mutation to maximise fairness and effectiveness. Eventually, it returns a set of Pareto optimal solutions, allowing the final stakeholders to choose the best one based on their needs. Method: We conduct an extensive empirical evaluation of FairRF against 26 different baselines in 11 different scenarios using five effectiveness and three fairness metrics. Additionally, we also include two variations of the fairness metrics for intersectional bias for a total of six definitions analysed. Result: Our results show that FairRF can significantly improve the fairness of base classifiers, while maintaining consistent prediction effectiveness. Additionally, FairRF provides a more consistent optimisation under all fairness definitions compared to state-of-the-art bias mitigation methods and overcomes the existing state-of-the-art approach for intersectional bias mitigation. Conclusions: FairRF is an effective approach for bias mitigation also allowing stakeholders to adapt the development of fair software systems based on their specific needs.

Figures (4)

• Figure 1: Example of Pareto front solutions returned by FairRF. The more solutions are closer to the upper-left corner of the plot, the better.
• Figure 2: Overview of the FairRF approach.
• Figure 3: RQ3: Pareto optimality of each approach considering each pair of effectiveness and fairness metrics.
• Figure 4: RQ4: Pareto optimality of each approach considering each pair of effectiveness and intersectional fairness metrics.