Can You Really Backdoor Federated Learning?

TL;DR

This work analyzes backdoor (targeted) attacks in federated learning, where adversaries aim to degrade specific inputs while preserving overall accuracy under realistic non-iid conditions. It formalizes a model update poisoning framework, comparing random and fixed-frequency attacker models, and introduces unconstrained and norm-bounded attack variants to obtain a backdoored model. The authors propose defenses—norm clipping and weak differential privacy—and show experimentally that these can substantially mitigate backdoor success with limited impact on the main task on EMNIST. They also provide open-source TensorFlow Federated implementations to encourage community development and standardized evaluation of attacks and defenses on federated datasets.

Abstract

The decentralized nature of federated learning makes detecting and defending against adversarial attacks a challenging task. This paper focuses on backdoor attacks in the federated learning setting, where the goal of the adversary is to reduce the performance of the model on targeted tasks while maintaining good performance on the main task. Unlike existing works, we allow non-malicious clients to have correctly labeled samples from the targeted tasks. We conduct a comprehensive study of backdoor attacks and defenses for the EMNIST dataset, a real-life, user-partitioned, and non-iid dataset. We observe that in the absence of defenses, the performance of the attack largely depends on the fraction of adversaries present and the "complexity'' of the targeted task. Moreover, we show that norm clipping and "weak'' differential privacy mitigate the attacks without hurting the overall performance. We have implemented the attacks and defenses in TensorFlow Federated (TFF), a TensorFlow framework for federated learning. In open-sourcing our code, our goal is to encourage researchers to contribute new attacks and defenses and evaluate them on standard federated datasets.

Figures (6)

• Figure 1: Unconstrained attack for fixed-frequency attacks (left column) and random sampling attack (right column) with different fractions of attackers. Green line is the cumulative mean for the backdoor accuracy.
• Figure 2: Constrained attack with norm bound 10 for fixed-frequency attacks (left column) and random sampling attack (right column) with different fractions of attackers. Green line is the cumulative mean for the backdoor accuracy.
• Figure 3: The Effect of Backdoor Size for Constrained Attack with Norm Bound 10.
• Figure 4: Effect of norm bounding and Gaussian noise. Dotted: main task. Solid: backdoor task.
• Figure 5: Unconstrained attack for fixed-frequency attacks (left column) and random sampling attack (right column) with different fractions of attackers. Green line is the cumulative mean for the backdoor accuracy.