Fake or Compromised? Making Sense of Malicious Clients in Federated Learning

TL;DR

A hybrid adversary model is presented, which lies in the middle of the spectrum of adversaries, where the adversary compromises a few clients, trains a generative model with their compromised samples, and generates new synthetic data to solve an optimization for a stronger attack against different robust aggregation rules.

Abstract

Federated learning (FL) is a distributed machine learning paradigm that enables training models on decentralized data. The field of FL security against poisoning attacks is plagued with confusion due to the proliferation of research that makes different assumptions about the capabilities of adversaries and the adversary models they operate under. Our work aims to clarify this confusion by presenting a comprehensive analysis of the various poisoning attacks and defensive aggregation rules (AGRs) proposed in the literature, and connecting them under a common framework. To connect existing adversary models, we present a hybrid adversary model, which lies in the middle of the spectrum of adversaries, where the adversary compromises a few clients, trains a generative (e.g., DDPM) model with their compromised samples, and generates new synthetic data to solve an optimization for a stronger (e.g., cheaper, more practical) attack against different robust aggregation rules. By presenting the spectrum of FL adversaries, we aim to provide practitioners and researchers with a clear understanding of the different types of threats they need to consider when designing FL systems, and identify areas where further research is needed.

Figures (6)

• Figure 1: Spectrum of the adversarial models that vary in the number of compromised clients and the number of fake clients injected into the FL system. (1) A scenario of fake clients may occur when FL applications are running on insecure FL platforms, or if we learn an FL model on Facebook or Twitter users, which can have a large number of fake accounts. In this scenario, the adversary can easily introduce fake clients, such as spam bots, into the FL ecosystem; these fake clients do not have any real data and can manipulate the updates they send to the central server. (2) A scenario of hybrid attack may occur when IoT devices participating in FL training, such as CCTV cameras or WiFi routers. An adversary can buy zombies from botnets for compromised and fake clients (more details in Section \ref{['sec:cost']}). (3) A scenario of compromised clients may occur in FL applications such as Google's Gboard, Apple's Siri, and Webank. In this scenario, the adversary may use sophisticated techniques such as social engineering, malware injection, or exploiting software vulnerabilities to compromise a small percentage of clients.
• Figure 2: Our novel hybrid attack pipeline: The adversary of hybrid attack lies in the middle of the spectrum of FL poisoning adversaries. The hybrid attack adversary compromises a few real FL clients, trains a denoising diffusion probability model (DDPM) on their real data, and generates new synthetic data to solve an optimization to generate malicious updates to mount strong model poisoning attacks against the target robust aggregation rules. Notably, if high-quality public data that mirrors real client data distribution is available, it can replace the initial data-gathering step in this process, although such data may not be readily available in proprietary contexts. Finally, the adversary shares the malicious update with the FL server via the compromised clients as well as (cheap to inject) fake clients.
• Figure 3: Number of samples for each label when the attacker compromised 0.1% (1 client), 0.3% (3 clients), and 0.5% (5 clients) in our data distribution (fixed through all the experiments) for learning CIFAR10 distributed over 1000 clients.
• Figure 4: Airplanes generated by DDPM and DCGAN using different percentages of compromised client's data in our hybrid attack.
• Figure 5: Attack impact ($I_{\theta}$) of the Norm-Bounding and Median aggregation rules in the presence of different adversaries. $\tau$ shows the $\ell_2$ threshold value that is used in Norm-Bounding aggreagtion. For hybrid attacks, we explore the impact of different numbers of compromised clients, specifically 0.5% (5 clients), 0.3% (3 clients), and 0.1% (1 client) in CIFAR10 experiments and 0.5% (17 clients), 0.3% (11 clients), and 0.1% (4 clients) in FEMNIST experiments.