Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Debiasing Machine Learning Models by Using Weakly Supervised Learning

Renan D. B. Brotto, Jean-Michel Loubes, Laurent Risser, Jean-Pierre Florens, Kenji Nose-Filho, João M. T. Romano

TL;DR

The paper tackles bias in real-valued predictions when the sensitive attribute is continuous by modeling endogeneity as $Y(\boldsymbol{x})=\varphi^{*}(\boldsymbol{x})+U(\boldsymbol{x})$ and solving an inverse-problem using instrumental variables within a weakly supervised learning framework. A two-stage debiasing pipeline first produces a local estimate from $Y$ and then refines it with a neural network trained with a small labeled set and a distributional fairness constraint via the 1-Wasserstein distance, thereby aligning the output distribution with an estimated fair distribution $\mathbb{P}(\varphi^{*})$. The authors provide theoretical bounds linking data-fidelity and distributional error to the labeled data, and demonstrate through 1D and 2D synthetic simulations, including time-varying fairness, that the method achieves strong debiasing with minimal supervision and robust adaptation to changing fairness notions. The work suggests practical potential for post-processing debiasing of black-box models in continuous-sensitive contexts, with future directions including real-world data testing and active labeling strategies.

Abstract

We tackle the problem of bias mitigation of algorithmic decisions in a setting where both the output of the algorithm and the sensitive variable are continuous. Most of prior work deals with discrete sensitive variables, meaning that the biases are measured for subgroups of persons defined by a label, leaving out important algorithmic bias cases, where the sensitive variable is continuous. Typical examples are unfair decisions made with respect to the age or the financial status. In our work, we then propose a bias mitigation strategy for continuous sensitive variables, based on the notion of endogeneity which comes from the field of econometrics. In addition to solve this new problem, our bias mitigation strategy is a weakly supervised learning method which requires that a small portion of the data can be measured in a fair manner. It is model agnostic, in the sense that it does not make any hypothesis on the prediction model. It also makes use of a reasonably large amount of input observations and their corresponding predictions. Only a small fraction of the true output predictions should be known. This therefore limits the need for expert interventions. Results obtained on synthetic data show the effectiveness of our approach for examples as close as possible to real-life applications in econometrics.

Debiasing Machine Learning Models by Using Weakly Supervised Learning

TL;DR

The paper tackles bias in real-valued predictions when the sensitive attribute is continuous by modeling endogeneity as and solving an inverse-problem using instrumental variables within a weakly supervised learning framework. A two-stage debiasing pipeline first produces a local estimate from and then refines it with a neural network trained with a small labeled set and a distributional fairness constraint via the 1-Wasserstein distance, thereby aligning the output distribution with an estimated fair distribution . The authors provide theoretical bounds linking data-fidelity and distributional error to the labeled data, and demonstrate through 1D and 2D synthetic simulations, including time-varying fairness, that the method achieves strong debiasing with minimal supervision and robust adaptation to changing fairness notions. The work suggests practical potential for post-processing debiasing of black-box models in continuous-sensitive contexts, with future directions including real-world data testing and active labeling strategies.

Abstract

We tackle the problem of bias mitigation of algorithmic decisions in a setting where both the output of the algorithm and the sensitive variable are continuous. Most of prior work deals with discrete sensitive variables, meaning that the biases are measured for subgroups of persons defined by a label, leaving out important algorithmic bias cases, where the sensitive variable is continuous. Typical examples are unfair decisions made with respect to the age or the financial status. In our work, we then propose a bias mitigation strategy for continuous sensitive variables, based on the notion of endogeneity which comes from the field of econometrics. In addition to solve this new problem, our bias mitigation strategy is a weakly supervised learning method which requires that a small portion of the data can be measured in a fair manner. It is model agnostic, in the sense that it does not make any hypothesis on the prediction model. It also makes use of a reasonably large amount of input observations and their corresponding predictions. Only a small fraction of the true output predictions should be known. This therefore limits the need for expert interventions. Results obtained on synthetic data show the effectiveness of our approach for examples as close as possible to real-life applications in econometrics.
Paper Structure (10 sections, 1 theorem, 29 equations, 25 figures, 1 table)

This paper contains 10 sections, 1 theorem, 29 equations, 25 figures, 1 table.

Table of Contents

  1. Introduction
  2. Theoretical Background
  3. Methodology
  4. Bias mitigation in supervised Machine Learning Models
  5. Theoretical Guarantees
  6. Numerical Simulations
  7. 1-Dimensional Signals
  8. 2-Dimensional Signals
  9. Time-Variant Fairness
  10. Conclusions

Key Result

Theorem 1

Let us consider the following cost function to be minimized and the sets Let $G_{\theta^{*}}$ be a minimizer of $J(\cdot|\lambda)$. Then it holds that for all $\theta \in \Theta_{L}$, and for all $\theta \in \Theta_{W}$,

Figures (25)

  • Figure 1: Data processing pipeline: we first perform an initial and simple estimation on $Y(\mathbf{x})$, generating the initial estimate $\tilde{\varphi}(\mathbf{x})$; then, we use a neural network to refine it, producing the final estimate $\hat{\varphi}(\mathbf{x})$.
  • Figure 2: Illustration of the permutation problem that arises from the minimization of the Wasserstein Distance. In both cases, we have the same score values for the red and the blue lines, but they are not properly sorted.
  • Figure 3: Weakly supervised learning: from all of the available data, only a small fraction has been labeled, as represented by the blue area. The remaining data --- gray area --- must be used in an unsupervised manner.
  • Figure 4: Instrumental Regression and Landweber Iteration - $k=2$.
  • Figure 5: Instrumental Regression and Landweber Iteration - $k=10$.
  • ...and 20 more figures

Theorems & Definitions (3)

  • Theorem 1
  • proof
  • Remark