Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Practical Framework for Privacy-Preserving and Byzantine-robust Federated Learning

Baolei Zhang, Minghong Fang, Zhuqing Liu, Biao Yi, Peizhao Zhou, Yuan Wang, Tong Li, Zheli Liu

TL;DR

The paper tackles the practical challenge of combining Byzantine-robust aggregation with privacy-preserving federated learning, addressing the high overhead of private distance computations. It introduces ABBR, a two-server framework that uses data-independent random projection to perform vector-wise filtering in a low-dimensional space, paired with adaptive tuning to minimize the impact of misclassified malicious updates. The authors provide an error analysis, outline an adaptive clipping strategy, and demonstrate substantial runtime and communication reductions while preserving Byzantine resilience across multiple datasets and models. This work advances the practicality of secure, robust federated learning by bridging theory and scalable real-world deployment.

Abstract

Federated Learning (FL) allows multiple clients to collaboratively train a model without sharing their private data. However, FL is vulnerable to Byzantine attacks, where adversaries manipulate client models to compromise the federated model, and privacy inference attacks, where adversaries exploit client models to infer private data. Existing defenses against both backdoor and privacy inference attacks introduce significant computational and communication overhead, creating a gap between theory and practice. To address this, we propose ABBR, a practical framework for Byzantine-robust and privacy-preserving FL. We are the first to utilize dimensionality reduction to speed up the private computation of complex filtering rules in privacy-preserving FL. Additionally, we analyze the accuracy loss of vector-wise filtering in low-dimensional space and introduce an adaptive tuning strategy to minimize the impact of malicious models that bypass filtering on the global model. We implement ABBR with state-of-the-art Byzantine-robust aggregation rules and evaluate it on public datasets, showing that it runs significantly faster, has minimal communication overhead, and maintains nearly the same Byzantine-resilience as the baselines.

Practical Framework for Privacy-Preserving and Byzantine-robust Federated Learning

TL;DR

The paper tackles the practical challenge of combining Byzantine-robust aggregation with privacy-preserving federated learning, addressing the high overhead of private distance computations. It introduces ABBR, a two-server framework that uses data-independent random projection to perform vector-wise filtering in a low-dimensional space, paired with adaptive tuning to minimize the impact of misclassified malicious updates. The authors provide an error analysis, outline an adaptive clipping strategy, and demonstrate substantial runtime and communication reductions while preserving Byzantine resilience across multiple datasets and models. This work advances the practicality of secure, robust federated learning by bridging theory and scalable real-world deployment.

Abstract

Federated Learning (FL) allows multiple clients to collaboratively train a model without sharing their private data. However, FL is vulnerable to Byzantine attacks, where adversaries manipulate client models to compromise the federated model, and privacy inference attacks, where adversaries exploit client models to infer private data. Existing defenses against both backdoor and privacy inference attacks introduce significant computational and communication overhead, creating a gap between theory and practice. To address this, we propose ABBR, a practical framework for Byzantine-robust and privacy-preserving FL. We are the first to utilize dimensionality reduction to speed up the private computation of complex filtering rules in privacy-preserving FL. Additionally, we analyze the accuracy loss of vector-wise filtering in low-dimensional space and introduce an adaptive tuning strategy to minimize the impact of malicious models that bypass filtering on the global model. We implement ABBR with state-of-the-art Byzantine-robust aggregation rules and evaluate it on public datasets, showing that it runs significantly faster, has minimal communication overhead, and maintains nearly the same Byzantine-resilience as the baselines.

Paper Structure

This paper contains 29 sections, 2 theorems, 21 equations, 8 figures, 7 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Federated Learning
  4. Byzantine-robust Aggregation Rule
  5. Secure Two-Party Computation Framework : ABY
  6. ABBR Framework
  7. Problem Setting
  8. Challenge
  9. System Overview
  10. Dimensionality Reduction
  11. Adaptive Tuning
  12. Error Analysis
  13. Tuning Strategy
  14. Evaluation
  15. Setup
  16. ...and 14 more sections

Key Result

Theorem 1

Suppose that the target dimensionality $k$ of low-dimensional models satisfies the Inequation Target_dimension and the distribution of the $d \times k$ projection matrix $\mathbf{P}$ satisfies the Equation Distribution, the error of Euclidean distance and cosine distance of any two local models in t where $L'_i$ and $L'_j$ are the low-dimensional local models after performing the dimensionality re

Figures (8)

  • Figure 1: Overview of ABBR in $t$-th iteration of FL.
  • Figure 2: Changes in the relative position of local models after being projected from original-dimensional space to low-dimensional space.
  • Figure 3: High-level idea of adaptive tuning strategy.
  • Figure 4: Impact of the percentage of Byzantine clients on CIFAR-10 dataset. (a)-(d): original vector-wise filterings, (e)-(f): ABBR versions of vector-wise filterings.
  • Figure 5: Impact of the degree of non-IID data (simulated by using Dirichlet distribution) on CIFAR-10 dataset. (a)-(d): original vector-wise filterings, (e)-(f): ABBR versions of vector-wise filterings.
  • ...and 3 more figures

Theorems & Definitions (4)

  • Theorem 1
  • Proof 1
  • Theorem 2
  • Proof 2