Transparency and Proportionality in Post-Processing Algorithmic Bias Correction
Juliett Suárez Ferreira, Marija Slavkovik, Jorge Casillas
TL;DR
The study tackles the problem that post-processing bias corrections can still yield unfair outcomes by focusing on the distribution of prediction flips across groups. It introduces a formal set of flip-centered proportionality metrics (e.g., FR, N_flips, DFR, FRD, HDI) and group-specific versions to quantify and compare how corrections affect privileged and unprivileged groups. Through a toy example, it demonstrates that a method can achieve traditional fairness metrics while concentrating harmful flips on a particular group, highlighting the need for transparency and consideration of proportionality in bias mitigation. The proposed methodology provides practitioners with diagnostics that complement standard fairness measures, promoting fairer and more justifiable post-processing interventions, and paves the way for broader routine adoption and extension to more complex settings.
Abstract
Algorithmic decision-making systems sometimes produce errors or skewed predictions toward a particular group, leading to unfair results. Debiasing practices, applied at different stages of the development of such systems, occasionally introduce new forms of unfairness or exacerbate existing inequalities. We focus on post-processing techniques that modify algorithmic predictions to achieve fairness in classification tasks, examining the unintended consequences of these interventions. To address this challenge, we develop a set of measures that quantify the disparity in the flips applied to the solution in the post-processing stage. The proposed measures will help practitioners: (1) assess the proportionality of the debiasing strategy used, (2) have transparency to explain the effects of the strategy in each group, and (3) based on those results, analyze the possibility of the use of some other approaches for bias mitigation or to solve the problem. We introduce a methodology for applying the proposed metrics during the post-processing stage and illustrate its practical application through an example. This example demonstrates how analyzing the proportionality of the debiasing strategy complements traditional fairness metrics, providing a deeper perspective to ensure fairer outcomes across all groups.