Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Causal Fairness under Unobserved Confounding: A Neural Sensitivity Framework

Maresa Schröder, Dennis Frauen, Stefan Feuerriegel

TL;DR

This work proposes a novel neural framework for learning fair predictions, which allows for worst-case guarantees of the extent to which causal fairness can be violated due to unobserved confounding, and is the first work to study causal fairness under unobvised confounding.

Abstract

Fairness for machine learning predictions is widely required in practice for legal, ethical, and societal reasons. Existing work typically focuses on settings without unobserved confounding, even though unobserved confounding can lead to severe violations of causal fairness and, thus, unfair predictions. In this work, we analyze the sensitivity of causal fairness to unobserved confounding. Our contributions are three-fold. First, we derive bounds for causal fairness metrics under different sources of unobserved confounding. This enables practitioners to examine the sensitivity of their machine learning models to unobserved confounding in fairness-critical applications. Second, we propose a novel neural framework for learning fair predictions, which allows us to offer worst-case guarantees of the extent to which causal fairness can be violated due to unobserved confounding. Third, we demonstrate the effectiveness of our framework in a series of experiments, including a real-world case study about predicting prison sentences. To the best of our knowledge, ours is the first work to study causal fairness under unobserved confounding. To this end, our work is of direct practical value as a refutation strategy to ensure the fairness of predictions in high-stakes applications.

Causal Fairness under Unobserved Confounding: A Neural Sensitivity Framework

TL;DR

This work proposes a novel neural framework for learning fair predictions, which allows for worst-case guarantees of the extent to which causal fairness can be violated due to unobserved confounding, and is the first work to study causal fairness under unobvised confounding.

Abstract

Fairness for machine learning predictions is widely required in practice for legal, ethical, and societal reasons. Existing work typically focuses on settings without unobserved confounding, even though unobserved confounding can lead to severe violations of causal fairness and, thus, unfair predictions. In this work, we analyze the sensitivity of causal fairness to unobserved confounding. Our contributions are three-fold. First, we derive bounds for causal fairness metrics under different sources of unobserved confounding. This enables practitioners to examine the sensitivity of their machine learning models to unobserved confounding in fairness-critical applications. Second, we propose a novel neural framework for learning fair predictions, which allows us to offer worst-case guarantees of the extent to which causal fairness can be violated due to unobserved confounding. Third, we demonstrate the effectiveness of our framework in a series of experiments, including a real-world case study about predicting prison sentences. To the best of our knowledge, ours is the first work to study causal fairness under unobserved confounding. To this end, our work is of direct practical value as a refutation strategy to ensure the fairness of predictions in high-stakes applications.
Paper Structure (40 sections, 9 theorems, 60 equations, 15 figures, 6 tables, 2 algorithms)

This paper contains 40 sections, 9 theorems, 60 equations, 15 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Setup
  4. Bounds for causal fairness under unobserved confounding
  5. Setting: Generalized marginal sensitivity model
  6. Counterfactual unnesting (Step 1)
  7. Sensitivity analysis to bound path-specific causal fairness (Step 2)
  8. Building a fair prediction model under unobserved confounding
  9. Experiments
  10. Datasets
  11. Results
  12. Discussion
  13. Notation
  14. Discussion of applications
  15. Examples: causal fairness with unobserved confounding
  16. ...and 25 more sections

Key Result

Lemma 1

$\mathrm{CF}_{\mathcal{C}}\in \{\mathrm{DE}, \mathrm{IE}, \mathrm{SE}\}$ can be defined as a monotonically increasing function $h$ over a sum of unidentifiable effects $e \in \mathcal{E}$ in the SCM $\mathcal{C}$, where $\mathcal{E}$ denotes the set of all effects in $\mathcal{C}$, and an identifiab

Figures (15)

  • Figure 1: Example: Causal graph for predicting prison sentences.
  • Figure 1: We make contributions to multiple literature streams. Level according to Pearl's causality ladder.
  • Figure 2: Causal graph of the Standard Fairness Model with different sources of unobserved confounding $U_{\mathrm{DE}}$, $U_{\mathrm{IE}}$, and $U_{\mathrm{SE}}$.
  • Figure 3: Our workflow for deriving bounds on causal fairness under unobserved confounding.
  • Figure 4: Validity of our bounds. Our bounds successfully contain the oracle effects for different confounding levels $\Phi$. Results for different sensitivity parameters $\Gamma_M$ and $\Gamma_Y = 1$.
  • ...and 10 more figures

Theorems & Definitions (19)

  • Definition 1: SCM
  • Definition 2: Generalized marginal sensitivity model (GMSM)
  • Lemma 1
  • proof
  • Remark 1
  • Theorem 1: Bounds on path-specific causal fairness
  • proof
  • Remark 2
  • Theorem 2
  • Theorem 3: Ancestral set factorization from Correa.2021
  • ...and 9 more