Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SkyMask: Attack-agnostic Robust Federated Learning with Fine-grained Learnable Masks

Peishen Yan, Hao Wang, Tao Song, Yang Hua, Ruhui Ma, Ningxin Hu, Mohammad R. Haghighat, Haibing Guan

TL;DR

SkyMask introduces a server-side, attack-agnostic defense for federated learning that uses parameter-level learnable masks trained on a small root dataset to detect and mitigate Byzantine attacks, including subtle fine-grained variants. The method employs a two-stage process: initialize and train per-client masks on the root data, then cluster masks with a Gaussian mixture model to separate benign from malicious updates before aggregation. Empirical results across Fashion-MNIST, CIFAR-10, and CIFAR-100 show SkyMask achieving up to 14% higher testing accuracy than state-of-the-art defenses and maintaining robustness even when up to 80% of clients are malicious. The approach is scalable, modular, and complements existing FL defenses, offering strong generalization and practical applicability in diverse, distributed settings.

Abstract

Federated Learning (FL) is becoming a popular paradigm for leveraging distributed data and preserving data privacy. However, due to the distributed characteristic, FL systems are vulnerable to Byzantine attacks that compromised clients attack the global model by uploading malicious model updates. With the development of layer-level and parameter-level fine-grained attacks, the attacks' stealthiness and effectiveness have been significantly improved. The existing defense mechanisms solely analyze the model-level statistics of individual model updates uploaded by clients to mitigate Byzantine attacks, which are ineffective against fine-grained attacks due to unawareness or overreaction. To address this problem, we propose SkyMask, a new attack-agnostic robust FL system that firstly leverages fine-grained learnable masks to identify malicious model updates at the parameter level. Specifically, the FL server freezes and multiplies the model updates uploaded by clients with the parameter-level masks, and trains the masks over a small clean dataset (i.e., root dataset) to learn the subtle difference between benign and malicious model updates in a high-dimension space. Our extensive experiments involve different models on three public datasets under state-of-the-art (SOTA) attacks, where the results show that SkyMask achieves up to 14% higher testing accuracy compared with SOTA defense strategies under the same attacks and successfully defends against attacks with malicious clients of a high fraction up to 80%. Code is available at https://github.com/KoalaYan/SkyMask.

SkyMask: Attack-agnostic Robust Federated Learning with Fine-grained Learnable Masks

TL;DR

SkyMask introduces a server-side, attack-agnostic defense for federated learning that uses parameter-level learnable masks trained on a small root dataset to detect and mitigate Byzantine attacks, including subtle fine-grained variants. The method employs a two-stage process: initialize and train per-client masks on the root data, then cluster masks with a Gaussian mixture model to separate benign from malicious updates before aggregation. Empirical results across Fashion-MNIST, CIFAR-10, and CIFAR-100 show SkyMask achieving up to 14% higher testing accuracy than state-of-the-art defenses and maintaining robustness even when up to 80% of clients are malicious. The approach is scalable, modular, and complements existing FL defenses, offering strong generalization and practical applicability in diverse, distributed settings.

Abstract

Federated Learning (FL) is becoming a popular paradigm for leveraging distributed data and preserving data privacy. However, due to the distributed characteristic, FL systems are vulnerable to Byzantine attacks that compromised clients attack the global model by uploading malicious model updates. With the development of layer-level and parameter-level fine-grained attacks, the attacks' stealthiness and effectiveness have been significantly improved. The existing defense mechanisms solely analyze the model-level statistics of individual model updates uploaded by clients to mitigate Byzantine attacks, which are ineffective against fine-grained attacks due to unawareness or overreaction. To address this problem, we propose SkyMask, a new attack-agnostic robust FL system that firstly leverages fine-grained learnable masks to identify malicious model updates at the parameter level. Specifically, the FL server freezes and multiplies the model updates uploaded by clients with the parameter-level masks, and trains the masks over a small clean dataset (i.e., root dataset) to learn the subtle difference between benign and malicious model updates in a high-dimension space. Our extensive experiments involve different models on three public datasets under state-of-the-art (SOTA) attacks, where the results show that SkyMask achieves up to 14% higher testing accuracy compared with SOTA defense strategies under the same attacks and successfully defends against attacks with malicious clients of a high fraction up to 80%. Code is available at https://github.com/KoalaYan/SkyMask.
Paper Structure (27 sections, 7 equations, 5 figures, 3 tables)

This paper contains 27 sections, 7 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminaries & Related Work
  3. Federated Learning
  4. Threat Model & Byzantine Attacks on FL
  5. Byzantine-robust FL Algorithms
  6. Root Dataset
  7. Methodology
  8. Overview
  9. SkyMask Algorithm
  10. Mask initialization and training.
  11. Mask clustering and classification.
  12. Why SkyMask Works
  13. SkyMask's Complexity
  14. Experiments
  15. Experiment Settings
  16. ...and 12 more sections

Figures (5)

  • Figure 1: Visualizing model updates and masks with PCA.
  • Figure 2: SkyMask's workflow.
  • Figure 3: The visualization of the training process of masks applied to malicious and benign models.
  • Figure 4: The impact of high fractions of malicious clients under fine-grained attacks.
  • Figure 5: The impact of detection failure on different malicious client detection methods.