A Data-Driven Defense against Edge-case Model Poisoning Attacks on Federated Learning

TL;DR

DataDefense addresses edge-case, targeted model poisoning in federated learning by introducing a defense dataset $D_d$ and learning two models: a poisoned-data detector $\\gamma(x,y;\\psi)$ and a client-importance model $\\mathcal{C}(\\phi_j;\\theta)$. The central server uses a weighted aggregation $ar{\\phi}^t$ with client weights derived from the CI model, while the PDD ranks defense samples and guides the partition into $D_{dc}$ and $D_{dp}$; both components are trained via alternating minimization across FL rounds. Experimental results across six diverse tasks show that DataDefense substantially reduces attack success rates compared to nine baselines, achieving strong robustness even with as few as five defense samples and under non-IID conditions. The approach offers practical and scalable defense for real-world FL deployments, with potential extensions to multiple concurrent backdoors and broader attack distributions.

Abstract

Federated Learning systems are increasingly subjected to a multitude of model poisoning attacks from clients. Among these, edge-case attacks that target a small fraction of the input space are nearly impossible to detect using existing defenses, leading to a high attack success rate. We propose an effective defense using an external defense dataset, which provides information about the attack target. The defense dataset contains a mix of poisoned and clean examples, with only a few known to be clean. The proposed method, DataDefense, uses this dataset to learn a poisoned data detector model which marks each example in the defense dataset as poisoned or clean. It also learns a client importance model that estimates the probability of a client update being malicious. The global model is then updated as a weighted average of the client models' updates. The poisoned data detector and the client importance model parameters are updated using an alternating minimization strategy over the Federated Learning rounds. Extensive experiments on standard attack scenarios demonstrate that DataDefense can defend against model poisoning attacks where other state-of-the-art defenses fail. In particular, DataDefense is able to reduce the attack success rate by at least ~ 40% on standard attack setups and by more than 80% on some setups. Furthermore, DataDefense requires very few defense examples (as few as five) to achieve a near-optimal reduction in attack success rate.

Figures (11)

• Figure 1: Performance comparison of DataDefense with Sparsefed panda2022sparsefed under PGD with/without model replacement attack for CIFAR-10 Southwest.
• Figure 2: (a) Percent of detected poison points in $D_d$ showing the effectiveness of $\psi$. (b) Analysis of client importance showing the effectiveness of $\theta$ under PGD with model replacement attack for CIFAR-10 Southwest
• Figure 3: Robustness of DataDefense under PGD with model replacement attack for CIFAR-10 Trigger Patch dataset with trigger patch size $14\times14$ after incomplete learning by halting updation of $\theta$ and $\psi$ after 100 rounds. (a) This led to a rapid increase in ASR to 100%. (b) and a positive client importance (CI) difference indicating higher CI attribution to the attacker.
• Figure 4: Label of a random image is flipped to bird
• Figure 5: Southwest airplanes labeled as “truck” to backdoor a CIFAR-10 classifier.