Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Local Layer-wise Differential Privacy in Federated Learning

Yunbo Li, Jiaping Gui, Fanchao Meng, Yue Wu

TL;DR

LaDP is proposed, a novel layer-wise adaptive noise injection mechanism for FL that optimizes privacy protection while preserving model accuracy and provides a rigorous theoretical analysis, proving that LaDP satisfies $(\epsilon, \delta)$-DP guarantees and converges under bounded noise.

Abstract

Federated Learning (FL) enables collaborative model training without direct data sharing, yet it remains vulnerable to privacy attacks such as model inversion and membership inference. Existing differential privacy (DP) solutions for FL often inject noise uniformly across the entire model, degrading utility while providing suboptimal privacy-utility tradeoffs. To address this, we propose LaDP, a novel layer-wise adaptive noise injection mechanism for FL that optimizes privacy protection while preserving model accuracy. LaDP leverages two key insights: (1) neural network layers contribute unevenly to model utility, and (2) layer-wise privacy leakage can be quantified via KL divergence between local and global model distributions. LaDP dynamically injects noise into selected layers based on their privacy sensitivity and importance to model performance. We provide a rigorous theoretical analysis, proving that LaDP satisfies $(ε, δ)$-DP guarantees and converges under bounded noise. Extensive experiments on CIFAR-10/100 datasets demonstrate that LaDP reduces noise injection by 46.14% on average compared to state-of-the-art (SOTA) methods while improving accuracy by 102.99%. Under the same privacy budget, LaDP outperforms SOTA solutions like Dynamic Privacy Allocation LDP and AdapLDP by 25.18% and 6.1% in accuracy, respectively. Additionally, LaDP robustly defends against reconstruction attacks, increasing the FID of the reconstructed private data by $>$12.84% compared to all baselines. Our work advances the practical deployment of privacy-preserving FL with minimal utility loss.

Local Layer-wise Differential Privacy in Federated Learning

TL;DR

LaDP is proposed, a novel layer-wise adaptive noise injection mechanism for FL that optimizes privacy protection while preserving model accuracy and provides a rigorous theoretical analysis, proving that LaDP satisfies -DP guarantees and converges under bounded noise.

Abstract

Federated Learning (FL) enables collaborative model training without direct data sharing, yet it remains vulnerable to privacy attacks such as model inversion and membership inference. Existing differential privacy (DP) solutions for FL often inject noise uniformly across the entire model, degrading utility while providing suboptimal privacy-utility tradeoffs. To address this, we propose LaDP, a novel layer-wise adaptive noise injection mechanism for FL that optimizes privacy protection while preserving model accuracy. LaDP leverages two key insights: (1) neural network layers contribute unevenly to model utility, and (2) layer-wise privacy leakage can be quantified via KL divergence between local and global model distributions. LaDP dynamically injects noise into selected layers based on their privacy sensitivity and importance to model performance. We provide a rigorous theoretical analysis, proving that LaDP satisfies -DP guarantees and converges under bounded noise. Extensive experiments on CIFAR-10/100 datasets demonstrate that LaDP reduces noise injection by 46.14% on average compared to state-of-the-art (SOTA) methods while improving accuracy by 102.99%. Under the same privacy budget, LaDP outperforms SOTA solutions like Dynamic Privacy Allocation LDP and AdapLDP by 25.18% and 6.1% in accuracy, respectively. Additionally, LaDP robustly defends against reconstruction attacks, increasing the FID of the reconstructed private data by 12.84% compared to all baselines. Our work advances the practical deployment of privacy-preserving FL with minimal utility loss.
Paper Structure (39 sections, 20 equations, 8 figures, 10 tables, 4 algorithms)

This paper contains 39 sections, 20 equations, 8 figures, 10 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Central Differential Privacy Federated Learning.
  4. Local Differential Privacy Federated Learning.
  5. Background
  6. Federated Learning
  7. Differential Privacy
  8. Threat Model
  9. Methodology
  10. System Overview
  11. System Design
  12. Layer Selection
  13. Privacy Estimation
  14. Noise Injection
  15. Theoretical Analysis
  16. ...and 24 more sections

Figures (8)

  • Figure 1: An FL model with an honest-but-curious client trying to infer private information from the global model. "Gen." and "Disc." denote Generator and Discriminator, respectively.
  • Figure 2: A visual representation of each module of LaDP-FL. The blue and red squares represent different levels of privacy information associated with different layers, resulting in different amounts of noise injection.
  • Figure 3: Illustration of KL divergence's relationship with privacy risk. (A) When $KL \approx 0$, the global model has a higher privacy risk because it closely resembles the local model. (B) When $KL \gg 0$, the local layer reveals little about the global mode due to the significant deviation.
  • Figure 4: The influence of the privacy budget $\epsilon$ on global accuracy and loss when using ResNet-18 on different datasets.
  • Figure 5: The comparison of accumulative privacy budget under different privacy budgets $\epsilon$ between LaDP-FL and baselines when training ResNet-18 on the CIFAR-10 dataset. Notably, the curves for Full DP and AdapLDP completely overlap; we distinguish them using distinct markers at varying intervals.
  • ...and 3 more figures

Theorems & Definitions (2)

  • proof
  • proof