Do Fairness Interventions Come at the Cost of Privacy: Evaluations for Binary Classifiers

TL;DR

The paper investigates whether in-processing fairness interventions for binary classifiers impact privacy, evaluating privacy leakage with membership inference attacks and attribute inference attacks. It finds that conventional attacks often become less effective on fair models, due in part to reduced sensitive content in features and lower confidence for majority data. However, it also uncovers a novel threat mechanism based on prediction gaps between biased and fair models, introducing FD-MIA and FD-AIA, which can enhance attack performance. Extensive experiments across three datasets, multiple fairness approaches, and attack types demonstrate both reduced leakage for standard attacks and the new privacy risks associated with fairness-induced disparities. The work highlights the need for thorough security evaluations when deploying fairness interventions and proposes defense strategies including restricting information access and differential privacy.

Abstract

While in-processing fairness approaches show promise in mitigating biased predictions, their potential impact on privacy leakage remains under-explored. We aim to address this gap by assessing the privacy risks of fairness-enhanced binary classifiers via membership inference attacks (MIAs) and attribute inference attacks (AIAs). Surprisingly, our results reveal that enhancing fairness does not necessarily lead to privacy compromises. For example, these fairness interventions exhibit increased resilience against MIAs and AIAs. This is because fairness interventions tend to remove sensitive information among extracted features and reduce confidence scores for the majority of training data for fairer predictions. However, during the evaluations, we uncover a potential threat mechanism that exploits prediction discrepancies between fair and biased models, leading to advanced attack results for both MIAs and AIAs. This mechanism reveals potent vulnerabilities of fair models and poses significant privacy risks of current fairness methods. Extensive experiments across multiple datasets, attack methods, and representative fairness approaches confirm our findings and demonstrate the efficacy of the uncovered mechanism. Our study exposes the under-explored privacy threats in fairness studies, advocating for thorough evaluations of potential security vulnerabilities before model deployments.

Figures (12)

• Figure 1: Fairness interventions increase loss values for the majority training data, leading to diminished attack successful rates for MIAs (Figure \ref{['fig:intro_mia']}). Meanwhile, compared to biased models, fair models show more resilience to AIAs under both Black and White-box settings (Figure \ref{['fig:intro_aia']}).
• Figure 2: Model privacy impact evaluation pipelines.
• Figure 3: Existing attacks (a) exhibit clear performance trade-offs between member and non-member data, each green circle represents attack accuracy on biased models and each blue circle represents attack accuracy on fair models; and (b) are inefficient in attacking hard examples in the low FPR region.
• Figure 4: Prediction score changes after applying fairness methods. The red lines in (a) and (b) indicate that the trained attack models infer sample membership with certain threshold values. (c) and (d) show the changes in terms of different subgroups.
• Figure 5: Histograms of prediction score distances between groups of member and non-member data for fair and biased models. We measure the distance with score value difference between the groups and present comparisons regarding (a) all data and (b) hard examples, where samples from the member and non-member data share similar scores.