Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Achieving Byzantine-Resilient Federated Learning via Layer-Adaptive Sparsified Model Aggregation

Jiahao Xu, Zikai Zhang, Rui Hu

TL;DR

This work addresses Byzantine resilience in Federated Learning by introducing LASA, a two-stage server-side defense that combines per-client Top-$k$ sparsification with layer-wise adaptive aggregation. By capturing both magnitude and direction at the layer level and using Median-based Z-score filtering, LASA mitigates the impact of malicious updates while preserving benign model utility, especially in non-IID settings. The authors formalize a $\kappa$-robustness criterion and prove LASA achieves tight robustness bounds, and they also establish $(f,R)$-Byzantine resilience for FL with LASA. Extensive experiments across IID and non-IID data, multiple attack methods, and varying sparsification and filtering parameters show LASA consistently outperforms state-of-the-art defenses in robustness and accuracy, while maintaining manageable computational costs. The approach offers practical, theoretically grounded protection for real-world FL deployments.

Abstract

Federated Learning (FL) enables multiple clients to collaboratively train a model without sharing their local data. Yet the FL system is vulnerable to well-designed Byzantine attacks, which aim to disrupt the model training process by uploading malicious model updates. Existing robust aggregation rule-based defense methods overlook the diversity of magnitude and direction across different layers of the model updates, resulting in limited robustness performance, particularly in non-IID settings. To address these challenges, we propose the Layer-Adaptive Sparsified Model Aggregation (LASA) approach, which combines pre-aggregation sparsification with layer-wise adaptive aggregation to improve robustness. Specifically, LASA includes a pre-aggregation sparsification module that sparsifies updates from each client before aggregation, reducing the impact of malicious parameters and minimizing the interference from less important parameters for the subsequent filtering process. Based on sparsified updates, a layer-wise adaptive filter then adaptively selects benign layers using both magnitude and direction metrics across all clients for aggregation. We provide the detailed theoretical robustness analysis of LASA and the resilience analysis for the FL integrated with LASA. Extensive experiments are conducted on various IID and non-IID datasets. The numerical results demonstrate the effectiveness of LASA. Code is available at \url{https://github.com/JiiahaoXU/LASA}.

Achieving Byzantine-Resilient Federated Learning via Layer-Adaptive Sparsified Model Aggregation

TL;DR

This work addresses Byzantine resilience in Federated Learning by introducing LASA, a two-stage server-side defense that combines per-client Top- sparsification with layer-wise adaptive aggregation. By capturing both magnitude and direction at the layer level and using Median-based Z-score filtering, LASA mitigates the impact of malicious updates while preserving benign model utility, especially in non-IID settings. The authors formalize a -robustness criterion and prove LASA achieves tight robustness bounds, and they also establish -Byzantine resilience for FL with LASA. Extensive experiments across IID and non-IID data, multiple attack methods, and varying sparsification and filtering parameters show LASA consistently outperforms state-of-the-art defenses in robustness and accuracy, while maintaining manageable computational costs. The approach offers practical, theoretically grounded protection for real-world FL deployments.

Abstract

Federated Learning (FL) enables multiple clients to collaboratively train a model without sharing their local data. Yet the FL system is vulnerable to well-designed Byzantine attacks, which aim to disrupt the model training process by uploading malicious model updates. Existing robust aggregation rule-based defense methods overlook the diversity of magnitude and direction across different layers of the model updates, resulting in limited robustness performance, particularly in non-IID settings. To address these challenges, we propose the Layer-Adaptive Sparsified Model Aggregation (LASA) approach, which combines pre-aggregation sparsification with layer-wise adaptive aggregation to improve robustness. Specifically, LASA includes a pre-aggregation sparsification module that sparsifies updates from each client before aggregation, reducing the impact of malicious parameters and minimizing the interference from less important parameters for the subsequent filtering process. Based on sparsified updates, a layer-wise adaptive filter then adaptively selects benign layers using both magnitude and direction metrics across all clients for aggregation. We provide the detailed theoretical robustness analysis of LASA and the resilience analysis for the FL integrated with LASA. Extensive experiments are conducted on various IID and non-IID datasets. The numerical results demonstrate the effectiveness of LASA. Code is available at \url{https://github.com/JiiahaoXU/LASA}.
Paper Structure (21 sections, 6 theorems, 43 equations, 3 figures, 6 tables, 1 algorithm)

This paper contains 21 sections, 6 theorems, 43 equations, 3 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Our Solution: LASA
  4. Robustness and Resilience Analysis of LASA
  5. Robustness analysis of LASA
  6. Resilience analysis of FL with LASA
  7. Evaluation
  8. Conclusion
  9. Appdenix
  10. Attack and defense models
  11. Experimental settings
  12. Evaluated attack methods
  13. Additional experimental results
  14. More results under various attack ratios
  15. Impact of sparsification level
  16. ...and 6 more sections

Key Result

Lemma 1

Under Assumption ass1- boundedsp, if $n \geq 1$ and $0 \leq f < n/2$, the proposed LASA method is a $\kappa$-robust aggregation rule with if the learning rate $\eta\leq 1/2\tau$ and the selection set satisfies $|\mathcal{S}^l| \geq n/2 - f,\forall l \in [L]$, $\tau$ is the number of local iteration. Here, $C_{\lambda_m}^2$ and $C^2$ represent the upper bound of the norm of malicious and benign up

Figures (3)

  • Figure 1: TPR, FPR, and Testing Accuracy (%) of LASA and SignGuard under ByzMean Attack on Shakespeare Dataset.
  • Figure 2: Testing Accuracy (%) of LASA SparseFed, DnC, and SignGuard under Various Attack Ratios on the Non-IID FEMNIST (upper) and Shakespeare (lower) Datasets.
  • Figure 3: Testing Accuracy of LASA, SignGuard, DnC and SparseFed under Various Attack Ratios in non-IID Settings.

Theorems & Definitions (18)

  • Definition 1: Top-$k$ sparsifier hu2023federated
  • Definition 2: Positive Direction Purity
  • Definition 3: MZ-score
  • Definition 4: $\kappa$-robustness
  • Lemma 1: $\kappa$-robustness of LASA
  • proof
  • Remark
  • Definition 5: $(f, R)$-Byzantine resilience kappaallouah2024robust
  • Theorem 1: $(f, R)$-Byzantine resilience of LASA
  • proof
  • ...and 8 more