Defending Against Sophisticated Poisoning Attacks with RL-based Aggregation in Federated Learning

TL;DR

AdaAggRL tackles the problem of defending FL against advanced poisoning attacks tailored for servers by learning and leveraging client data distribution stability. The method builds a pipeline of distribution learning, MMD-based similarity cues, and TD3-based policy learning to adaptively assign aggregation weights, suppressing malicious updates while preserving benign contributions. Empirical results across MNIST, F-MNIST, EMNIST, and CIFAR-10 show AdaAggRL outperforms existing defenses under multiple attack types, including RL-based attackers, and remains effective under high non-i.i.d. and attacker fractions. This approach offers a practical, computation-aware defense that does not require root data and enhances the reliability of FL in adversarial environments.

Abstract

Federated learning is highly susceptible to model poisoning attacks, especially those meticulously crafted for servers. Traditional defense methods mainly focus on updating assessments or robust aggregation against manually crafted myopic attacks. When facing advanced attacks, their defense stability is notably insufficient. Therefore, it is imperative to develop adaptive defenses against such advanced poisoning attacks. We find that benign clients exhibit significantly higher data distribution stability than malicious clients in federated learning in both CV and NLP tasks. Therefore, the malicious clients can be recognized by observing the stability of their data distribution. In this paper, we propose AdaAggRL, an RL-based Adaptive Aggregation method, to defend against sophisticated poisoning attacks. Specifically, we first utilize distribution learning to simulate the clients' data distributions. Then, we use the maximum mean discrepancy (MMD) to calculate the pairwise similarity of the current local model data distribution, its historical data distribution, and global model data distribution. Finally, we use policy learning to adaptively determine the aggregation weights based on the above similarities. Experiments on four real-world datasets demonstrate that the proposed defense model significantly outperforms widely adopted defense models for sophisticated attacks.

Figures (13)

• Figure 1: The statistical results of the similarity between the current client data distribution and its historical data distributions under four types of attacks vary with the training epochs on MNIST dataset. The x-axis denotes the number of client update rounds, and the y-axis represents the similarity between the current and its historical data distributions.
• Figure 2: The statistical results of the similarity between the current client data distribution and the global model data distribution under attacks vary with epochs on MNIST.
• Figure 3: An overview of our AdaAggRL
• Figure 4: The testing accuracy variation of the global model on Cifar10 dataset under four attacks.
• Figure 5: The testing accuracy of FL methods on MNIST-0.5 under LMP and EB as the proportion of malicious clients increases (a-b). The defense performance of AdaAggRL on MNIST-0.5 compared to the case where $S_{cl}$ is not considered under LMP (c) and the case where $S_{cg}$ and $S_{lg}$ are not considered under EB (d).