Fairness Is Not Just Ethical: Performance Trade-Off via Data Correlation Tuning to Mitigate Bias in ML Software

TL;DR

This work reframes software fairness as a core software quality attribute and introduces Correlation Tuning (CoT), a model-agnostic pre-processing method that mitigates bias by adjusting data correlations using the Phi-coefficient and multi-objective optimization. CoT targets both sensitive and non-sensitive attribute-driven biases, supports single and multiple sensitive attributes, and yields substantial improvements in unprivileged group performance while reducing SPD, AOD, and EOD with minimal accuracy loss. Across ten benchmark tasks and multiple models, CoT achieves state-of-the-art fairness improvements and favorable performance-fairness trade-offs, including strong intersectional bias mitigation. The method demonstrates efficiency, scalability, and robustness, with plans to extend to NLP and LLM contexts and to public release of data and code for further research and practical deployment in diverse software systems.

Abstract

Traditional software fairness research typically emphasizes ethical and social imperatives, neglecting that fairness fundamentally represents a core software quality issue arising directly from performance disparities across sensitive user groups. Recognizing fairness explicitly as a software quality dimension yields practical benefits beyond ethical considerations, notably improved predictive performance for unprivileged groups, enhanced out-of-distribution generalization, and increased geographic transferability in real-world deployments. Nevertheless, existing bias mitigation methods face a critical dilemma: while pre-processing methods offer broad applicability across model types, they generally fall short in effectiveness compared to post-processing techniques. To overcome this challenge, we propose Correlation Tuning (CoT), a novel pre-processing approach designed to mitigate bias by adjusting data correlations. Specifically, CoT introduces the Phi-coefficient, an intuitive correlation measure, to systematically quantify correlation between sensitive attributes and labels, and employs multi-objective optimization to address the proxy biases. Extensive evaluations demonstrate that CoT increases the true positive rate of unprivileged groups by an average of 17.5% and reduces three key bias metrics, including statistical parity difference (SPD), average odds difference (AOD), and equal opportunity difference (EOD), by more than 50% on average. CoT outperforms state-of-the-art methods by three and ten percentage points in single attribute and multiple attributes scenarios, respectively. We will publicly release our experimental results and source code to facilitate future research.

Figures (2)

• Figure 1: Overview of CoT
• Figure 2: (RQ3) Effectiveness level distribution of existing methods and CoT in the performance-fairness trade-off. Both CoT-Phi and CoT-Opt outperform state-of-the-art methods in balancing model performance and fairness. Specifically, CoT-Phi surpasses the Fairea-trade-off baseline in 79.2% of cases (win-win + good), and CoT-Opt achieves this in 79.9%, while the state-of-the-art achieves this in 76.7% of cases.