Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Random Gradient Masking as a Defensive Measure to Deep Leakage in Federated Learning

Joon Kim, Sejin Park

TL;DR

Deep Leakage from Gradients (DLG) challenges privacy in Federated Learning by enabling reconstruction of private data from gradients. The paper introduces Random Gradient Masking and compares it with Noising, Clipping, and Pruning across MNIST, CIFAR-10, and lfw, using SSIM-based leakage assessments and convergence tests. Key findings show Masking and Clipping defend against DLG with minimal impact on training accuracy, while Noising and Pruning can degrade performance; a BatchNorm-specific masking workaround expands applicability. The work provides a practical privacy-utility toolkit for FL, demonstrating scalable defenses that preserve model utility while reducing leakage risk.

Abstract

Federated Learning(FL), in theory, preserves privacy of individual clients' data while producing quality machine learning models. However, attacks such as Deep Leakage from Gradients(DLG) severely question the practicality of FL. In this paper, we empirically evaluate the efficacy of four defensive methods against DLG: Masking, Clipping, Pruning, and Noising. Masking, while only previously studied as a way to compress information during parameter transfer, shows surprisingly robust defensive utility when compared to the other three established methods. Our experimentation is two-fold. We first evaluate the minimum hyperparameter threshold for each method across MNIST, CIFAR-10, and lfw datasets. Then, we train FL clients with each method and their minimum threshold values to investigate the trade-off between DLG defense and training performance. Results reveal that Masking and Clipping show near to none degradation in performance while obfuscating enough information to effectively defend against DLG.

Random Gradient Masking as a Defensive Measure to Deep Leakage in Federated Learning

TL;DR

Deep Leakage from Gradients (DLG) challenges privacy in Federated Learning by enabling reconstruction of private data from gradients. The paper introduces Random Gradient Masking and compares it with Noising, Clipping, and Pruning across MNIST, CIFAR-10, and lfw, using SSIM-based leakage assessments and convergence tests. Key findings show Masking and Clipping defend against DLG with minimal impact on training accuracy, while Noising and Pruning can degrade performance; a BatchNorm-specific masking workaround expands applicability. The work provides a practical privacy-utility toolkit for FL, demonstrating scalable defenses that preserve model utility while reducing leakage risk.

Abstract

Federated Learning(FL), in theory, preserves privacy of individual clients' data while producing quality machine learning models. However, attacks such as Deep Leakage from Gradients(DLG) severely question the practicality of FL. In this paper, we empirically evaluate the efficacy of four defensive methods against DLG: Masking, Clipping, Pruning, and Noising. Masking, while only previously studied as a way to compress information during parameter transfer, shows surprisingly robust defensive utility when compared to the other three established methods. Our experimentation is two-fold. We first evaluate the minimum hyperparameter threshold for each method across MNIST, CIFAR-10, and lfw datasets. Then, we train FL clients with each method and their minimum threshold values to investigate the trade-off between DLG defense and training performance. Results reveal that Masking and Clipping show near to none degradation in performance while obfuscating enough information to effectively defend against DLG.
Paper Structure (15 sections, 1 equation, 5 figures, 1 table, 2 algorithms)

This paper contains 15 sections, 1 equation, 5 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. The Naive FL Model
  4. Deep Leakage from Gradients
  5. Methodology
  6. Client-Side Obfuscation
  7. BatchNorm in Masking
  8. Experiment Design
  9. Threshold Measurement
  10. Performance Evaluation
  11. Discussion
  12. Threshold Measurement
  13. Performance Evaluation
  14. Conclusion
  15. Full Algorithm of Obfuscated FL

Figures (5)

  • Figure 1: Flowchart Diagram for BatchNorm Circumvention in Masking
  • Figure 2: iDLG that are leaked with brightness adjustments. The leftmost image is the leaked image, and the middle image is the one scoring best in SSIM compared to the original image, out of all brightness-adjusted images. The leaked image at first seems obfuscated enough, but with adjustments, it is leaked.
  • Figure 3: iDLG that are robust even with brightness adjustments. The obfuscation methods are identical to Figure \ref{['fig:leakage_with_adjustments_leaked']}, but with hyperparameters intensified to obfuscate more information against the third party.
  • Figure 4: iDLG without any obfuscation
  • Figure 5: Performance Evaluation of various obfuscation algorithms with CIFAR-10 for Median(Left) and Mean(Right), IID(Up) and Non-IID(Down).