Privacy at a Price: Exploring its Dual Impact on AI Fairness
Mengmeng Yang, Ming Ding, Youyang Qu, Wei Ni, David Smith, Thierry Rakotoarivelo
TL;DR
This paper investigates how differential privacy (DP) affects fairness in AI, revealing a non-monotonic relationship where subgroup accuracy gaps first widen with increasing privacy noise and eventually diminish at high privacy levels. It analyzes two DP perturbation strategies—output perturbation and gradient perturbation (DP-SGD)—across multiple datasets (Adult, Bank, MNIST) and demonstrates that gradient clipping can postpone and mitigate fairness degradation. The findings show that excessive DP noise can homogenize performance across groups, while inappropriate clipping can exacerbate disparities, highlighting a nuanced privacy-fairness trade-off. The work informs privacy-preserving AI design under regulatory contexts and calls for careful calibration of DP parameters to balance privacy safeguards with equitable outcomes.
Abstract
The worldwide adoption of machine learning (ML) and deep learning models, particularly in critical sectors, such as healthcare and finance, presents substantial challenges in maintaining individual privacy and fairness. These two elements are vital to a trustworthy environment for learning systems. While numerous studies have concentrated on protecting individual privacy through differential privacy (DP) mechanisms, emerging research indicates that differential privacy in machine learning models can unequally impact separate demographic subgroups regarding prediction accuracy. This leads to a fairness concern, and manifests as biased performance. Although the prevailing view is that enhancing privacy intensifies fairness disparities, a smaller, yet significant, subset of research suggests the opposite view. In this article, with extensive evaluation results, we demonstrate that the impact of differential privacy on fairness is not monotonous. Instead, we observe that the accuracy disparity initially grows as more DP noise (enhanced privacy) is added to the ML process, but subsequently diminishes at higher privacy levels with even more noise. Moreover, implementing gradient clipping in the differentially private stochastic gradient descent ML method can mitigate the negative impact of DP noise on fairness. This mitigation is achieved by moderating the disparity growth through a lower clipping threshold.