Does Privacy Always Harm Fairness? Data-Dependent Trade-offs via Chernoff Information Neural Estimation

TL;DR

This work investigates how fairness, privacy, and accuracy interact in machine learning and reveals that the relationship is data-dependent rather than universal. It introduces Noisy Chernoff Difference (and its noisy variant $\ ilde{CD}_{\eta^2}$) to quantify how group separability—and thus potential fairness gaps—evolve under privacy-preserving noise, establishing three Gaussian-regime behaviors where privacy can harm, be neutral, or even improve fairness. The paper then develops Chernoff Information Neural Estimation (CINE), a neural-density-ratio-based estimator, to compute Chernoff Information from real data and validates it on synthetic Gaussian mixtures and real datasets (Adult, MNIST, HSLS). Across mixtures and real data, the results show data distributions largely determine the fairness–privacy–accuracy trade-offs, with the Noisy Chernoff Difference acting as a proxy for the slope of fairness–accuracy curves and guiding when privacy may yield “free fairness.” These insights advance a data-distribution-aware view of designing private, fair ML systems and offer a practical estimation tool for practitioners to assess these trade-offs on their data.

Abstract

Fairness and privacy are two vital pillars of trustworthy machine learning. Despite extensive research on these individual topics, the relationship between fairness and privacy has received significantly less attention. In this paper, we utilize the information-theoretic measure Chernoff Information to highlight the data-dependent nature of the relationship among the triad of fairness, privacy, and accuracy. We first define Noisy Chernoff Difference, a tool that allows us to analyze the relationship among the triad simultaneously. We then show that for synthetic data, this value behaves in 3 distinct ways (depending on the distribution of the data). We highlight the data distributions involved in these cases and explore their fairness and privacy implications. Additionally, we show that Noisy Chernoff Difference acts as a proxy for the steepness of the fairness-accuracy curves. Finally, we propose a method for estimating Chernoff Information on data from unknown distributions and utilize this framework to examine the triad dynamic on real datasets. This work builds towards a unified understanding of the fairness-privacy-accuracy relationship and highlights its data-dependent nature.

Figures (15)

• Figure 1: (Case 1) (a) $\widetilde{\text{CD}}_{\eta^2}$ increases until the maximum point, which represents the worst fairness-accuracy trade-off we encounter by adding noise. (b) The slopes of the fairness-accuracy plots increase as we add noise until the $\widetilde{\text{CD}}_{\eta^2}$ reaches the maximum. Parameters used here are: $\mu_0 = 0, \mu_1=16.5,\sigma = 2.43$ and $\zeta_0 = 0.5, \zeta_1 =3.8, \tau = 0.55$. (Case 2) (c) $\widetilde{\text{CD}}_{\eta^2}$ decays until it reaches 0. It then reflects back and follows a similar pattern to Case 1. The maximum occurs at a large $\eta^2$ (6.86) value and thus is not plotted (Appendix \ref{['ssec:case2_moredetail']}). (d) The steepness of the fairness-accuracy plots decreases as we add noise and CD decreases. The intersecting lines show that in some cases, we can achieve better fairness for the same accuracy when we add noise. Parameters used here are: $\mu_0 = -4.2, \mu_1=1.3,\sigma = 3$ and $\zeta_0 = 0.3, \zeta_1 =2.7, \tau = 0.25$. (Case 3) (e) $\widetilde{\text{CD}}_{\eta^2}$ decays steadily over the entire positive $\eta^2$ regime. (f) Fairness-accuracy curves become flatter as $\widetilde{\text{CD}}_{\eta^2}$ decreases with the increasing noise. Parameters used here are: $\mu_0 = -4.2, \mu_1=1.3,\sigma = 0.85$ and $\zeta_0 = 0.6, \zeta_1 =1.6, \tau = 0.6$.
• Figure 2: (Algorithm 2 performance on Gaussian Data) (a) In 2D, using distributions $\mathcal{N}(\textbf{0}, \sigma^2\mathbf{I})$ and $\mathcal{N}(\textbf{1}, \sigma^2\mathbf{I})$ Chernoff Information estimates remain accurate for different values of $\sigma^2$. (b) In 5D, using distributions $\mathcal{N}(\textbf{0}, \frac{1}{2}\mathbf{I})$ and $\mathcal{N}(\textbf{1},\textbf{I})$ Chernoff Information estimation remains accurate, even as noise $(\eta^2)$ is added. (c) Using distributions $\mathcal{N}(\textbf{0}, \frac{1}{2}\mathbf{I})$ and $\mathcal{N}(\textbf{1},\textbf{I})$, estimations are accurate in lower dimensions before degrading.
• Figure 3: (Mixture 1) (a) $\widetilde{\text{CD}}_{\eta^2}$ spikes as $\eta^2$ increases. (b) Fairness-Accuracy curve becomes more steep resembling fairness-accuracy curves from Case 1 from isotropic Gaussians. (Mixture 2) (c) $\widetilde{\text{CD}}_{\eta^2}$ decreasing to 0 (d) Fairness-Accuracy curves flattening resembling fairness-accuracy curves from Case 2 from isotropic Gaussians. (Adult) (e) $\widetilde{\text{CD}}_{\eta^2}$ Remains Flat for Adult dataset. (f) Fairness-Accuracy slopes remain stable for Adult dataset. (MNIST) (g) $\widetilde{\text{CD}}_{\eta^2}$ decreases for experiments on MNIST representations. (h) Fairness-Accuracy Curves Flatten for MNIST representations.
• Figure D.1: (Case 1: Privacy Hurts Fairness)$\mu_0 = 0, \mu_1 = 16.5, \sigma = 2.43$, $\zeta_0 = 0.5, \zeta_1 = 3.8, \tau = 0.55$. (a) Fairness-Accuracy Curve. (b) Log Fairness-Accuracy Curve. We observe a steepening effect.
• Figure D.2: (Case 2: Privacy Can Give Free Fairness)$\mu_0 = -4.2, \mu_1 = 1.3, \sigma = 3$, $\zeta_0 = 0.3, \zeta_1 = 2.7, \tau = 0.25$. (a) Fairness-Accuracy Curve. (b) Log Fairness-Accuracy Curve. We observe a flattening effect.

Theorems & Definitions (43)

• Definition 1: Differential Privacy
• Lemma 1
• Definition 2: Chernoff Information chernoff1952measure
• Lemma 2: nielsen2011
• Definition 3: Chernoff Difference
• Remark 1: Chernoff Difference as a Data Fairness Measure
• Definition 4: Noisy Chernoff Difference
• Lemma 3
• Lemma 4
• Theorem 1