When Fairness Meets Privacy: Exploring Privacy Threats in Fair Binary Classifiers via Membership Inference Attacks

TL;DR

This study examines how fairness interventions in binary classifiers affect privacy, specifically through membership inference attacks (MIAs). It reveals that traditional score-based MIAs underperform on fairness-enhanced models due to attack models collapsing to simple thresholds, while fairness can widen the discrimination gap between member and non-member predictions. The authors introduce FD-MIA, a novel attack that exploits prediction discrepancies between biased and fairness-enforced models, and demonstrate its superior efficacy across diverse datasets, attack types, and fairness methods. They also propose mitigation strategies, including restricting output access and applying differential privacy (DP-SGD), showing that FD-MIA remains a strong threat even under DP, underscoring the need for careful design and evaluation of trustworthy, privacy-preserving fair systems.

Abstract

Previous studies have developed fairness methods for biased models that exhibit discriminatory behaviors towards specific subgroups. While these models have shown promise in achieving fair predictions, recent research has identified their potential vulnerability to score-based membership inference attacks (MIAs). In these attacks, adversaries can infer whether a particular data sample was used during training by analyzing the model's prediction scores. However, our investigations reveal that these score-based MIAs are ineffective when targeting fairness-enhanced models in binary classifications. The attack models trained to launch the MIAs degrade into simplistic threshold models, resulting in lower attack performance. Meanwhile, we observe that fairness methods often lead to prediction performance degradation for the majority subgroups of the training data. This raises the barrier to successful attacks and widens the prediction gaps between member and non-member data. Building upon these insights, we propose an efficient MIA method against fairness-enhanced models based on fairness discrepancy results (FD-MIA). It leverages the difference in the predictions from both the original and fairness-enhanced models and exploits the observed prediction gaps as attack clues. We also explore potential strategies for mitigating privacy leakages. Extensive experiments validate our findings and demonstrate the efficacy of the proposed method.

Figures (10)

• Figure 1: Fairness interventions tend to (a) increase the losses and (b) decrease attack success rates for most training samples. We generate the plots with $100$ runs and report the mean loss value and mean attack success rate for the training sample.
• Figure 2: Attacking fair and biased models with MIAs. We first attack them separately and then compare the results to explore the privacy impact of fairness interventions.
• Figure 3: Existing attacks (a) exhibit clear performance trade-offs between member and non-member data, and (b) are inefficient in attacking hard examples in the low FPR regime.
• Figure 4: Prediction score changes after applying fairness methods. The red lines in (a) and (b) indicate that the trained attack models infer sample membership with certain threshold values. (c) and (d) show the changes in terms of different subgroups.
• Figure 5: Histograms of prediction score distances between member and non-member data. The plots show enlarged distance when considering both fair and biased models.