Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping

Xiaoyu Cao, Minghong Fang, Jia Liu, Neil Zhenqiang Gong

TL;DR

FLTrust introduces server-side trust bootstrapping for Byzantine-robust federated learning by employing a small, clean root dataset to train a server model. It leverages a new aggregation rule that uses ReLU-clipped cosine similarities to assign trust to client updates and normalizes their magnitudes to the server's, then averages updates weighted by trust scores. The method defends against data-poisoning and local model poisoning—including adaptive attacks—while preserving fidelity close to FedAvg under no attacks and maintaining efficiency. The authors provide formal convergence-like guarantees and extensive empirical validation across six datasets, demonstrating robust performance with a small root dataset and highlighting practical considerations and limitations.

Abstract

Byzantine-robust federated learning aims to enable a service provider to learn an accurate global model when a bounded number of clients are malicious. The key idea of existing Byzantine-robust federated learning methods is that the service provider performs statistical analysis among the clients' local model updates and removes suspicious ones, before aggregating them to update the global model. However, malicious clients can still corrupt the global models in these methods via sending carefully crafted local model updates to the service provider. The fundamental reason is that there is no root of trust in existing federated learning methods. In this work, we bridge the gap via proposing FLTrust, a new federated learning method in which the service provider itself bootstraps trust. In particular, the service provider itself collects a clean small training dataset (called root dataset) for the learning task and the service provider maintains a model (called server model) based on it to bootstrap trust. In each iteration, the service provider first assigns a trust score to each local model update from the clients, where a local model update has a lower trust score if its direction deviates more from the direction of the server model update. Then, the service provider normalizes the magnitudes of the local model updates such that they lie in the same hyper-sphere as the server model update in the vector space. Our normalization limits the impact of malicious local model updates with large magnitudes. Finally, the service provider computes the average of the normalized local model updates weighted by their trust scores as a global model update, which is used to update the global model. Our extensive evaluations on six datasets from different domains show that our FLTrust is secure against both existing attacks and strong adaptive attacks.

FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping

TL;DR

FLTrust introduces server-side trust bootstrapping for Byzantine-robust federated learning by employing a small, clean root dataset to train a server model. It leverages a new aggregation rule that uses ReLU-clipped cosine similarities to assign trust to client updates and normalizes their magnitudes to the server's, then averages updates weighted by trust scores. The method defends against data-poisoning and local model poisoning—including adaptive attacks—while preserving fidelity close to FedAvg under no attacks and maintaining efficiency. The authors provide formal convergence-like guarantees and extensive empirical validation across six datasets, demonstrating robust performance with a small root dataset and highlighting practical considerations and limitations.

Abstract

Byzantine-robust federated learning aims to enable a service provider to learn an accurate global model when a bounded number of clients are malicious. The key idea of existing Byzantine-robust federated learning methods is that the service provider performs statistical analysis among the clients' local model updates and removes suspicious ones, before aggregating them to update the global model. However, malicious clients can still corrupt the global models in these methods via sending carefully crafted local model updates to the service provider. The fundamental reason is that there is no root of trust in existing federated learning methods. In this work, we bridge the gap via proposing FLTrust, a new federated learning method in which the service provider itself bootstraps trust. In particular, the service provider itself collects a clean small training dataset (called root dataset) for the learning task and the service provider maintains a model (called server model) based on it to bootstrap trust. In each iteration, the service provider first assigns a trust score to each local model update from the clients, where a local model update has a lower trust score if its direction deviates more from the direction of the server model update. Then, the service provider normalizes the magnitudes of the local model updates such that they lie in the same hyper-sphere as the server model update in the vector space. Our normalization limits the impact of malicious local model updates with large magnitudes. Finally, the service provider computes the average of the normalized local model updates weighted by their trust scores as a global model update, which is used to update the global model. Our extensive evaluations on six datasets from different domains show that our FLTrust is secure against both existing attacks and strong adaptive attacks.

Paper Structure

This paper contains 25 sections, 5 theorems, 43 equations, 7 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Background on Federated Learning (FL)
  4. FedAvg
  5. Byzantine-robust Aggregation Rules
  6. Poisoning Attacks to Federated Learning
  7. Problem Setup
  8. Our FLTrust
  9. Overview of FLTrust
  10. Our New Aggregation Rule
  11. Complete FLTrust Algorithm
  12. Formal Security Analysis
  13. Adaptive Attacks
  14. Local Model Poisoning Attack Framework
  15. Adaptive Attack to Our FLTrust
  16. ...and 10 more sections

Key Result

Theorem 1

Suppose Assumption assumption_1-assumption_3 hold and FLTrust uses $R_l=1$ and $\beta=1$. For an arbitrary number of malicious clients, the difference between the global model learnt by FLTrust and the optimal global model $\bm{w}^*$ under no attacks is bounded. Formally, we have the following with where $\bm{w}^t$ is the global model in the $t$th iteration, $\rho = 1- \left( \sqrt {1 - {\mu^2}/

Figures (7)

  • Figure 1: Illustration of the three steps in FL.
  • Figure 2: Illustration of our aggregation rule, which is applied in each iteration of FLTrust.
  • Figure 3: The training error rates vs. the number of iterations for FLTrust under different attacks and FedAvg without attacks on MNIST-0.5.
  • Figure 4: Impact of the root dataset size on FLTrust under different attacks for MNIST-0.5.
  • Figure 5: Impact of the total number of clients on the testing error rates of different FL methods under different attacks ((a)-(c)) and the attack success rates of the Scaling attacks, where MNIST-0.5 is used. The testing error rates of all the compared FL methods are similar and small under the Scaling attacks, which we omit for simplicity.
  • ...and 2 more figures

Theorems & Definitions (10)

  • Theorem 1
  • proof
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • Lemma 4
  • proof