BadSampler: Harnessing the Power of Catastrophic Forgetting to Poison Byzantine-robust Federated Learning
Yi Liu, Cong Wang, Xingliang Yuan
TL;DR
The paper investigates poisoning in Byzantine-robust Federated Learning by exploiting catastrophic forgetting through a clean-label data poisoning attack. It introduces BadSampler, which uses two adaptive sampling strategies—Top-$\kappa$ sampling and meta-sampling—optimized via Soft Actor-Critic to maximize generalization error while keeping training error low, all under a realistic threat model with $M \le 10\%$ compromised clients. The authors provide a theoretical upper bound on attack-induced gradient shifts and show favorable complexity, and they validate the approach on Fashion-MNIST and CIFAR-10 across multiple defenses, demonstrating significant reductions in accuracy. The work highlights a practical vulnerability in production FL and motivates the development of defenses that monitor training dynamics and generalization drift beyond traditional anomaly detection.
Abstract
Federated Learning (FL) is susceptible to poisoning attacks, wherein compromised clients manipulate the global model by modifying local datasets or sending manipulated model updates. Experienced defenders can readily detect and mitigate the poisoning effects of malicious behaviors using Byzantine-robust aggregation rules. However, the exploration of poisoning attacks in scenarios where such behaviors are absent remains largely unexplored for Byzantine-robust FL. This paper addresses the challenging problem of poisoning Byzantine-robust FL by introducing catastrophic forgetting. To fill this gap, we first formally define generalization error and establish its connection to catastrophic forgetting, paving the way for the development of a clean-label data poisoning attack named BadSampler. This attack leverages only clean-label data (i.e., without poisoned data) to poison Byzantine-robust FL and requires the adversary to selectively sample training data with high loss to feed model training and maximize the model's generalization error. We formulate the attack as an optimization problem and present two elegant adversarial sampling strategies, Top-$κ$ sampling, and meta-sampling, to approximately solve it. Additionally, our formal error upper bound and time complexity analysis demonstrate that our design can preserve attack utility with high efficiency. Extensive evaluations on two real-world datasets illustrate the effectiveness and performance of our proposed attacks.