Fairness, Accuracy, and Unreliable Data

TL;DR

This work addresses how standard empirical risk minimization can fail when data exhibit unreliable properties such as fairness bias, sequential screening, or adversarial perturbations. It develops mathematical models to study three axes—fairness under biased data, multi-stage screening with fairness constraints, and adversarial robustness—proposing interventions like Equal Opportunity, re-weighting, and randomized hypothesis expansions to recover or preserve performance. The thesis provides formal theorems and proofs delineating when fairness constraints improve or harm performance, exact and approximate algorithms for optimizing fairness-constrained pipelines, and defense strategies against strategic manipulation and malicious noise, validated by synthetic and semi-synthetic experiments. Collectively, these results offer practical guidance for designing reliable AI systems in high-stakes settings, along with rigorous boundaries on what fairness interventions can and cannot achieve under data unreliability. Theoretical insights are complemented by algorithmic frameworks (DP-based and FPTAS approaches) and empirical validation to guide practitioners in choosing distribution-aware fairness, screening, and robustness strategies.

Abstract

This thesis investigates three areas targeted at improving the reliability of machine learning; fairness in machine learning, strategic classification, and algorithmic robustness. Each of these domains has special properties or structure that can complicate learning. A theme throughout this thesis is thinking about ways in which a `plain' empirical risk minimization algorithm will be misleading or ineffective because of a mis-match between classical learning theory assumptions and specific properties of some data distribution in the wild. Theoretical understanding in eachof these domains can help guide best practices and allow for the design of effective, reliable, and robust systems.

Figures (18)

• Figure 1: The schematic on the left displays data points with $p=1/2$, $h^{*}_{B}$ as a hyperplane, and $\eta=1/3$. The schematic on the right displays data drawn from the same distribution subject to the Under-Representation Bias with $\beta_{POS}=1/3$. Now there are more negative examples than positive examples above the hyperplane so the lowest error hypothesis classifies all examples on the right as negative.
• Figure 2: This figure indicates the parameter region such that Equal Opportunity Constrained ERM recovers $h^*$ under the Under-Representation Bias Model and is a visualization of Equation \ref{['Equal Opportunityineq']}. $r=1/3$ and $p=1/2$. We label each pair $(\eta, \beta)$ with blue if it satisfies the inequality and red otherwise. This plot shows how smaller $\eta$ means we can recover from lower $\beta$. Blue means $h^*$ is recovered. The dashed black line indicates the boundary between recovering $h^*$ and failing to recover $h^*$.
• Figure 3: Differences between $h_B$ and $h_{B}^*$ measured with probabilities in the true data distribution (before the effects of the bias model).
• Figure 4: Fully Synthetic Experiment Showing Accuracy Loss. Vertical dashed blue line is when $\eta = (1-\eta)\beta$, e.g. to the left of this line.
• Figure 5: This figure has higher values of $\eta$ and $r$ so we are not in the Strong Recovery Regime, meaning the theoretical bounds for Equal Opportunity recovery in this bias model is greater than zero. We again see close alignment with our theoretical bounds.

Theorems & Definitions (107)

• Definition 1
• Definition 2
• Theorem 3
• Lemma 4
• Lemma 5
• Lemma 6
• Lemma 7
• Theorem 8
• Definition 9: $\mathop{\mathrm{\mathcal{P}\mathcal{Q}(\mathcal{H})}}\nolimits$
• Definition 10