Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Byzantine-Resilient Federated Learning via Distributed Optimization

Yufei Xia, Wenrui Yu, Qiongxiu Li

TL;DR

Byzantine attacks threaten Federated Learning by injecting malicious updates that degrade model quality. The authors propose a distributed optimization approach using the Primal-Dual Method of Multipliers (PDMM) to enforce consensus via constraints like $B_{i|j} w_i + B_{j|i} w_j = 0$, with updates such as $w_i^{(t+1)} = argmin_{w_i} [ f_i(w_i) + (c/2) || w_i - y^{(t)}_{s|i} ||^2 ]$ and $w_s^{(t+1)} = (1/N) \sum_i y^{(t+1)}_{i|s}$. Empirically, PDMM-based FL yields higher accuracy, faster convergence, and reduced variance under bit-flipping and Gaussian-noise attacks across CFL and DFL on MNIST, FashionMNIST, and Olivetti. This protocol-level robustness broadens defenses beyond aggregation-based schemes, highlighting distributed optimization as a principled route for secure FL in adversarial environments.

Abstract

Byzantine attacks present a critical challenge to Federated Learning (FL), where malicious participants can disrupt the training process, degrade model accuracy, and compromise system reliability. Traditional FL frameworks typically rely on aggregation-based protocols for model updates, leaving them vulnerable to sophisticated adversarial strategies. In this paper, we demonstrate that distributed optimization offers a principled and robust alternative to aggregation-centric methods. Specifically, we show that the Primal-Dual Method of Multipliers (PDMM) inherently mitigates Byzantine impacts by leveraging its fault-tolerant consensus mechanism. Through extensive experiments on three datasets (MNIST, FashionMNIST, and Olivetti), under various attack scenarios including bit-flipping and Gaussian noise injection, we validate the superior resilience of distributed optimization protocols. Compared to traditional aggregation-centric approaches, PDMM achieves higher model utility, faster convergence, and improved stability. Our results highlight the effectiveness of distributed optimization in defending against Byzantine threats, paving the way for more secure and resilient federated learning systems.

Byzantine-Resilient Federated Learning via Distributed Optimization

TL;DR

Byzantine attacks threaten Federated Learning by injecting malicious updates that degrade model quality. The authors propose a distributed optimization approach using the Primal-Dual Method of Multipliers (PDMM) to enforce consensus via constraints like , with updates such as and . Empirically, PDMM-based FL yields higher accuracy, faster convergence, and reduced variance under bit-flipping and Gaussian-noise attacks across CFL and DFL on MNIST, FashionMNIST, and Olivetti. This protocol-level robustness broadens defenses beyond aggregation-based schemes, highlighting distributed optimization as a principled route for secure FL in adversarial environments.

Abstract

Byzantine attacks present a critical challenge to Federated Learning (FL), where malicious participants can disrupt the training process, degrade model accuracy, and compromise system reliability. Traditional FL frameworks typically rely on aggregation-based protocols for model updates, leaving them vulnerable to sophisticated adversarial strategies. In this paper, we demonstrate that distributed optimization offers a principled and robust alternative to aggregation-centric methods. Specifically, we show that the Primal-Dual Method of Multipliers (PDMM) inherently mitigates Byzantine impacts by leveraging its fault-tolerant consensus mechanism. Through extensive experiments on three datasets (MNIST, FashionMNIST, and Olivetti), under various attack scenarios including bit-flipping and Gaussian noise injection, we validate the superior resilience of distributed optimization protocols. Compared to traditional aggregation-centric approaches, PDMM achieves higher model utility, faster convergence, and improved stability. Our results highlight the effectiveness of distributed optimization in defending against Byzantine threats, paving the way for more secure and resilient federated learning systems.

Paper Structure

This paper contains 14 sections, 5 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Aggregation based Federated Learning
  4. Byzantine Attacks
  5. Methodology
  6. Distributed Optimization in FL
  7. PDMM based FL
  8. Analysis of Byzantine robustness
  9. Numerical Results
  10. Experimental setup
  11. Performance comparison under bit flipping Attacks
  12. Performance comparison under Gaussian noise attacks
  13. Performance superiority is independent of topology type
  14. Conclusion

Figures (2)

  • Figure 1: Byzantine robustness comparison of PDMM based FL protocols over traditional FedAvg based protocols against big flipping attack over three datasets and two types of topologies (CFL v.s. DFL).
  • Figure 2: Byzantine robustness comparison of PDMM based FL protocols over traditional FedAvg based protocols against Gaussian noise attack over three datasets and two types topologies (CFL v.s., DFL).