Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Understanding Adversarial Transferability in Federated Learning

Yijiang Li, Ying Gao, Haohan Wang

TL;DR

Surprisingly, FL systems show a higher level of robustness than their centralized counterparts, especially when both systems are equally good at handling regular, non-malicious data.

Abstract

We investigate a specific security risk in FL: a group of malicious clients has impacted the model during training by disguising their identities and acting as benign clients but later switching to an adversarial role. They use their data, which was part of the training set, to train a substitute model and conduct transferable adversarial attacks against the federated model. This type of attack is subtle and hard to detect because these clients initially appear to be benign. The key question we address is: How robust is the FL system to such covert attacks, especially compared to traditional centralized learning systems? We empirically show that the proposed attack imposes a high security risk to current FL systems. By using only 3\% of the client's data, we achieve the highest attack rate of over 80\%. To further offer a full understanding of the challenges the FL system faces in transferable attacks, we provide a comprehensive analysis over the transfer robustness of FL across a spectrum of configurations. Surprisingly, FL systems show a higher level of robustness than their centralized counterparts, especially when both systems are equally good at handling regular, non-malicious data. We attribute this increased robustness to two main factors: 1) Decentralized Data Training: Each client trains the model on its own data, reducing the overall impact of any single malicious client. 2) Model Update Averaging: The updates from each client are averaged together, further diluting any malicious alterations. Both practical experiments and theoretical analysis support our conclusions. This research not only sheds light on the resilience of FL systems against hidden attacks but also raises important considerations for their future application and development.

Towards Understanding Adversarial Transferability in Federated Learning

TL;DR

Surprisingly, FL systems show a higher level of robustness than their centralized counterparts, especially when both systems are equally good at handling regular, non-malicious data.

Abstract

We investigate a specific security risk in FL: a group of malicious clients has impacted the model during training by disguising their identities and acting as benign clients but later switching to an adversarial role. They use their data, which was part of the training set, to train a substitute model and conduct transferable adversarial attacks against the federated model. This type of attack is subtle and hard to detect because these clients initially appear to be benign. The key question we address is: How robust is the FL system to such covert attacks, especially compared to traditional centralized learning systems? We empirically show that the proposed attack imposes a high security risk to current FL systems. By using only 3\% of the client's data, we achieve the highest attack rate of over 80\%. To further offer a full understanding of the challenges the FL system faces in transferable attacks, we provide a comprehensive analysis over the transfer robustness of FL across a spectrum of configurations. Surprisingly, FL systems show a higher level of robustness than their centralized counterparts, especially when both systems are equally good at handling regular, non-malicious data. We attribute this increased robustness to two main factors: 1) Decentralized Data Training: Each client trains the model on its own data, reducing the overall impact of any single malicious client. 2) Model Update Averaging: The updates from each client are averaged together, further diluting any malicious alterations. Both practical experiments and theoretical analysis support our conclusions. This research not only sheds light on the resilience of FL systems against hidden attacks but also raises important considerations for their future application and development.
Paper Structure (36 sections, 5 theorems, 24 equations, 20 figures, 4 tables, 1 algorithm)

This paper contains 36 sections, 5 theorems, 24 equations, 20 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Adversarial Robustness
  4. The Security and Robustness of Federated Learning
  5. Poisoning and Backdoor Attack.
  6. Transfer Attack.
  7. Investigation Setup and Research Goals
  8. Experiment Setup
  9. Adversarial Transferability in Federated Learning
  10. Experiments
  11. Robustness with Comparable Accuracy
  12. Robustness Against Transfer Attack
  13. Transfer Attack with limited data
  14. Two intrinsic properties contributing to transfer robustness
  15. Decentralized training and data Heterogeneity
  16. ...and 21 more sections

Key Result

Theorem 6.1

With Assumptions 2-8, we have: where $L, \mu, \sigma_k, G$ are defined in the assumptions, $\kappa=\frac{L}{\mu}$, $\gamma = \max\{8\kappa, E\}$ , $B=\sum_{k=1}^N p_k^2 \sigma_k^2 + 6L\Gamma + 8(E-1)^2G^2$ and $C = \frac{4}{K}E^2G^2$. $\theta_1$ is the parameter after one step update of SGD. $\theta_\star$ is the optimal paramete

Figures (20)

  • Figure 1: Attack with data from a limited number of users. (a) We show the transfer rate of our attack with ResNet50 on the CIFAR10 dataset. We additionally experiment with attackers of different architectures. (b) experiment with CNN on CIFAR10 dataset. (c) we boost the performance of our attack with standard augmentation and pretraining techniques. (d) we perform our attack on different training stages. (e) we provide experiments on easier scenarios with 10 and 30 users, which further demonstrate the threat our attack poses.
  • Figure 2: (a) Comparison with query-based black-box attack. (b) Attack with the surrogate model trained with the same distribution (no participation in the FL training) (c) Attack with the surrogate model trained with similar knowledge, i.e. CIFAR100.
  • Figure 3: T.Rate vs. data of different heterogeneity and dispersion degree. (a): top 3 are results of ResNet50; (b): bottom 3 are results of CNN; Left: T.Rate as a function of the number of users in federated training; Middle: T.Rate as a function of dirichlet alpha; Right:T.Rate as a function of unbalanced sgm.
  • Figure 4: Transfer rate v.s. maximum number of classes per client; (a): Results of ResNet50 on CIFAR10 dataset; (b): Results of CNN on CIFAR10 dataset; It is noteworthy to observe that the transfer rate in the 10-user setting persistently surpasses that in the 100-user setting. This further substantiates the proposition that more decentralized training leads to lower adversarial transferability for the federated model.
  • Figure 5: Additional experiments on SVHN, CIFAR100 and ImageNet200. (a) Our attack with ResNet18 on SVHN (first row), ResNet50 on CIFAR100 (second row) and ResNet50 on ImageNet200 (third row). (b) How decentralization relates to the transfer robustness of FL model on SVHN (first row), CIFAR100 (second row) and ImageNet200 (third row) datasets. (c) How heterogeneity relates to the transfer robustness of FL model on SVHN (first row), CIFAR100 (second row) and ImageNet200 (third row) datasets. (d) How averaging operation relates to the transfer robustness of FL model on SVHN dataset (first row), CIFAR100 (second row) and ImageNet200 (third row) datasets.
  • ...and 15 more figures

Theorems & Definitions (10)

  • Theorem 6.1
  • Corollary 6.2
  • Remark 6.3
  • Remark 6.4
  • Theorem 6.5
  • Remark 6.6
  • Lemma D.1
  • Lemma D.2
  • proof
  • proof