Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Seesaw Model Attack Algorithm for Distributed Learning

Kun Yang, Tianyi Luo, Yanjie Dong, Aohan Li

TL;DR

This work contributes to the design of attack strategies by initially highlighting the limitations of finite-norm attacks, and introduces the seesaw attack, which has been demonstrated to be more effective than the finite-norm attack.

Abstract

We investigate the Byzantine attack problem within the context of model training in distributed learning systems. While ensuring the convergence of current model training processes, common solvers (e.g. SGD, Adam, RMSProp, etc.) can be easily compromised by malicious nodes in these systems. Consequently, the training process may either converge slowly or even diverge. To develop effective secure distributed learning solvers, it is crucial to first examine attack methods to assess the robustness of these solvers. In this work, we contribute to the design of attack strategies by initially highlighting the limitations of finite-norm attacks. We then introduce the seesaw attack, which has been demonstrated to be more effective than the finite-norm attack. Through numerical experiments, we evaluate the efficacy of the seesaw attack across various gradient aggregation rules.

A Seesaw Model Attack Algorithm for Distributed Learning

TL;DR

This work contributes to the design of attack strategies by initially highlighting the limitations of finite-norm attacks, and introduces the seesaw attack, which has been demonstrated to be more effective than the finite-norm attack.

Abstract

We investigate the Byzantine attack problem within the context of model training in distributed learning systems. While ensuring the convergence of current model training processes, common solvers (e.g. SGD, Adam, RMSProp, etc.) can be easily compromised by malicious nodes in these systems. Consequently, the training process may either converge slowly or even diverge. To develop effective secure distributed learning solvers, it is crucial to first examine attack methods to assess the robustness of these solvers. In this work, we contribute to the design of attack strategies by initially highlighting the limitations of finite-norm attacks. We then introduce the seesaw attack, which has been demonstrated to be more effective than the finite-norm attack. Through numerical experiments, we evaluate the efficacy of the seesaw attack across various gradient aggregation rules.
Paper Structure (27 sections, 3 equations, 7 figures)

This paper contains 27 sections, 3 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Contributions
  3. Federated Learning Gradient Aggregation
  4. Federated Learning Model with Byzantine Nodes
  5. Gradient Aggregation Rules (GAR)
  6. Weighted Average
  7. Trimmed Mean
  8. Median
  9. Krum
  10. FABA
  11. Geometric Median with Cosine Similarity
  12. Directional Deviation Attack
  13. Analysis of Directional Deviation Attack
  14. Seesaw Attack
  15. Hierarchical Clustering Krum
  16. ...and 12 more sections

Figures (7)

  • Figure 1: Neural Network Architecture.
  • Figure 2: Comparison of the defense effects of different aggregation strategies under seesaw attacks
  • Figure 3: Comparison of the defense effects of different aggregation strategies under limited norm attacks
  • Figure 4: Comparison of defense effects under different attacks with average aggregation. The severity of the Seesaw attack is slightly lower than that of the limited norm attack.
  • Figure 5: Comparison of defense effects under different attacks with Krum aggregation. The severity of the Seesaw attack is significantly higher than that of the limited norm attack
  • ...and 2 more figures