Fairness-Constrained Optimization Attack in Federated Learning
Harsh Kasyap, Minghong Fang, Zhuqing Liu, Carsten Maple, Somanath Tripathy
TL;DR
This work identifies a vulnerability in federated learning where a malicious client can deliberately inflate fairness loss to worsen subgroup fairness without harming overall accuracy. It formulates a fairness-constrained optimization attack using DP-based constraints and a regularization parameter $\lambda$, leveraging a cost-sensitive, reductions-based framework to perturb the local training objective while maintaining predictive performance. The approach employs a DP matrix $Q$ and moment vector $\eta(f)$ to define $M_*$, with gradient estimation via finite differences to handle non-differentiable fairness metrics, and scales to multiple demographic groups. Empirical results across five datasets show the attack can raise demographic parity and equalized odds disparities substantially (up to around 90% in some settings) and often evades detection by Byzantine-robust and fairness-aware aggregators, highlighting a critical need for stronger defenses against fairness-targeted poisoning in FL.
Abstract
Federated learning (FL) is a privacy-preserving machine learning technique that facilitates collaboration among participants across demographics. FL enables model sharing, while restricting the movement of data. Since FL provides participants with independence over their training data, it becomes susceptible to poisoning attacks. Such collaboration also propagates bias among the participants, even unintentionally, due to different data distribution or historical bias present in the data. This paper proposes an intentional fairness attack, where a client maliciously sends a biased model, by increasing the fairness loss while training, even considering homogeneous data distribution. The fairness loss is calculated by solving an optimization problem for fairness metrics such as demographic parity and equalized odds. The attack is insidious and hard to detect, as it maintains global accuracy even after increasing the bias. We evaluate our attack against the state-of-the-art Byzantine-robust and fairness-aware aggregation schemes over different datasets, in various settings. The empirical results demonstrate the attack efficacy by increasing the bias up to 90\%, even in the presence of a single malicious client in the FL system.