Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Masked Unfairness: Hiding Causality within Zero ATE

Zou Yang, Sophia Xiao, Bijan Mazaheri

Abstract

Recent work has proposed powerful frameworks, rooted in causal theory, to quantify fairness. Causal inference has primarily emphasized the detection of \emph{average} treatment effects (ATEs), and subsequent notions of fairness have inherited this focus. In this paper, we build on previous concerns about regulation based on averages. In particular, we formulate the "causal masking problem" as a linear program that optimizes an alternative objective, such as maximizing profit or minimizing crime, while retaining a zero ATE (i.e., the ATE between a protected attribute and a decision). By studying the capabilities and limitations of causal masking, we show that optimization under ATE-based regulation may induce significant unequal treatment. We demonstrate that the divergence between true and causally masked fairness is driven by confounding, underscoring the importance of full conditional-independence testing when assessing fairness. Finally, we discuss statistical and information-theoretic limitations that make causally masked solutions very difficult to detect, allowing them to persist for long periods. These results argue that we must regulate fairness at the model-level, rather than at the decision level.

Masked Unfairness: Hiding Causality within Zero ATE

Abstract

Recent work has proposed powerful frameworks, rooted in causal theory, to quantify fairness. Causal inference has primarily emphasized the detection of \emph{average} treatment effects (ATEs), and subsequent notions of fairness have inherited this focus. In this paper, we build on previous concerns about regulation based on averages. In particular, we formulate the "causal masking problem" as a linear program that optimizes an alternative objective, such as maximizing profit or minimizing crime, while retaining a zero ATE (i.e., the ATE between a protected attribute and a decision). By studying the capabilities and limitations of causal masking, we show that optimization under ATE-based regulation may induce significant unequal treatment. We demonstrate that the divergence between true and causally masked fairness is driven by confounding, underscoring the importance of full conditional-independence testing when assessing fairness. Finally, we discuss statistical and information-theoretic limitations that make causally masked solutions very difficult to detect, allowing them to persist for long periods. These results argue that we must regulate fairness at the model-level, rather than at the decision level.
Paper Structure (38 sections, 11 theorems, 26 equations, 4 figures, 1 table)

This paper contains 38 sections, 11 theorems, 26 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Summary of Contributions
  3. Related Works
  4. Formalizing Fairness.
  5. Optimization and Fairness.
  6. Fairness in Predictions.
  7. Computing HTEs.
  8. Preliminaries
  9. Notation.
  10. ATEs and CATEs.
  11. Causal Graphs.
  12. Motivating Example
  13. Setup Distribution.
  14. ATE as a Measure of Fairness.
  15. Evading ATE Detection.
  16. ...and 23 more sections

Key Result

Proposition 4.1

The "exploit" approach optimally uses $X, P$ without further constraints: The optimal policy participates greedily with nonzero $\alpha_{x, p}$ in $(x_1^*, p_1^*)$ with the highest possible reward function, $(x_1^*, p_1^*) = \arg\max_{x,p} \gamma(x,p)$, then the second highest $(x_2^*, p_2^*)$, and so on until the participation rate is exhausted.

Figures (4)

  • Figure 1: Two causal diagrams depicting the dependencies between the random variables in our setting. The undirected edge $X - P$ captures optionality in $X \leftarrow P$, or $X \rightarrow P$.
  • Figure 2: Two relaxations that allow improvements from causal masking (on the left), and one where there is no improvement from fairness (on the right).
  • Figure 3: Synthetic data experiments showing that relaxing masking and fairness yield tempting performance improvements.
  • Figure 4: Simulations showing the average (+- standard error) sample size needed to reject each null hypothesis for all three strategies.

Theorems & Definitions (20)

  • Proposition 4.1: Optimal Exploit Policy
  • Proposition 4.2: Optimal $\varepsilon$-Fair Policy
  • Proposition 4.3: Optimal $\varepsilon$-Masking Policy
  • Definition 5.1
  • Theorem 5.2: The Gap between Masking and Fairness
  • Theorem 5.3: Necessary Conditions for a Performance Gap
  • Theorem 5.4: Sufficient Conditions for a Generic Gap
  • proof
  • proof
  • proof
  • ...and 10 more