Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How to Defend Against Large-scale Model Poisoning Attacks in Federated Learning: A Vertical Solution

Jinbo Wang, Ruijin Wang, Fengli Zhang

TL;DR

This work proposes VERT, which uses global communication rounds as the vertical axis, trains a predictor using historical gradients information to predict user gradients, and compares the similarity with actual user gradients to precisely and efficiently select the optimal aggregation gradients.

Abstract

Federated learning (FL) is vulnerable to model poisoning attacks due to its distributed nature. The current defenses start from all user gradients (model updates) in each communication round and solve for the optimal aggregation gradients (horizontal solution). This horizontal solution will completely fail when facing large-scale (>50%) model poisoning attacks. In this work, based on the key insight that the convergence process of the model is a highly predictable process, we break away from the traditional horizontal solution of defense and innovatively transform the problem of solving the optimal aggregation gradients into a vertical solution problem. We propose VERT, which uses global communication rounds as the vertical axis, trains a predictor using historical gradients information to predict user gradients, and compares the similarity with actual user gradients to precisely and efficiently select the optimal aggregation gradients. In order to reduce the computational complexity of VERT, we design a low dimensional vector projector to project the user gradients to a computationally acceptable length, and then perform subsequent predictor training and prediction tasks. Exhaustive experiments show that VERT is efficient and scalable, exhibiting excellent large-scale (>=80%) model poisoning defense effects under different FL scenarios. In addition, we can design projector with different structures for different model structures to adapt to aggregation servers with different computing power.

How to Defend Against Large-scale Model Poisoning Attacks in Federated Learning: A Vertical Solution

TL;DR

This work proposes VERT, which uses global communication rounds as the vertical axis, trains a predictor using historical gradients information to predict user gradients, and compares the similarity with actual user gradients to precisely and efficiently select the optimal aggregation gradients.

Abstract

Federated learning (FL) is vulnerable to model poisoning attacks due to its distributed nature. The current defenses start from all user gradients (model updates) in each communication round and solve for the optimal aggregation gradients (horizontal solution). This horizontal solution will completely fail when facing large-scale (>50%) model poisoning attacks. In this work, based on the key insight that the convergence process of the model is a highly predictable process, we break away from the traditional horizontal solution of defense and innovatively transform the problem of solving the optimal aggregation gradients into a vertical solution problem. We propose VERT, which uses global communication rounds as the vertical axis, trains a predictor using historical gradients information to predict user gradients, and compares the similarity with actual user gradients to precisely and efficiently select the optimal aggregation gradients. In order to reduce the computational complexity of VERT, we design a low dimensional vector projector to project the user gradients to a computationally acceptable length, and then perform subsequent predictor training and prediction tasks. Exhaustive experiments show that VERT is efficient and scalable, exhibiting excellent large-scale (>=80%) model poisoning defense effects under different FL scenarios. In addition, we can design projector with different structures for different model structures to adapt to aggregation servers with different computing power.

Paper Structure

This paper contains 29 sections, 20 equations, 10 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Federated Learning and Model poisoning Attacks
  4. The predictability of user gradients in FL system
  5. Problem Formulation
  6. Optimization Objective
  7. Threat Model
  8. Proposed Method
  9. Rethinking the Defense Against Large-scale Model Poisoning Attacks in FL Systems from a Vertical Perspective
  10. Vertical Solution: VERT
  11. Integration of User Gradient and Global Gradient
  12. Prediction of User Gradient
  13. Reduce Memory Overhead: Low Dimensional Vector Projector
  14. Theoretical Analysis
  15. Computational Complexity Analysis
  16. ...and 14 more sections

Figures (10)

  • Figure 1: The direction of user gradients in all communication rounds in the FL system.
  • Figure 2: VERT framework. During the training phase, VERT utilizes historical gradients information to train the coefficient matrix, projector, and predictor, enabling the predictor to predict user gradients precisely. In the prediction stage, VERT inputs the gradient information from the previous round into the projector and predictor, calculates the cosine similarity between the prediction gradient and the actual gradient.
  • Figure 3: The performance of VERT in predicting user gradients in the face of large-scale GN attacks in IID scenarios.
  • Figure 4: The performance of VERT in predicting user gradients in the face of large-scale GN attacks in non-IID scenarios.
  • Figure 5: The performance of VERT in predicting user gradients in the face of large-scale MR attacks in IID scenarios.
  • ...and 5 more figures

Theorems & Definitions (3)

  • Definition 4.1: Horizontal solution
  • Definition 4.2: Malicious gradients dominance problem
  • Definition 4.3: Vertical solution