Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Local Differential Privacy for Federated Learning with Fixed Memory Usage and Per-Client Privacy

Rouzbeh Behnia, Jeremiah Birrell, Arman Riasi, Reza Ebrahimi, Kaushik Dutta, Thang Hoang

TL;DR

The paper addresses privacy risks in federated learning, where client updates and the global model can leak information. It introduces L-RDP, a fixed-memory local-DP accountant for FL that uses fixed-size minibatches to stabilize memory usage and provides per-client privacy guarantees under asynchronous participation, complemented by MPC-based model verification to protect model integrity. The approach is implemented in the Flower FL framework, with open-source artifacts forthcoming. Experiments demonstrate substantial memory savings (from up to 46 GB to about 14 GB) with negligible utility loss (around 1% relative drop) across image and NLP tasks at various privacy budgets, indicating practical viability for privacy-preserving, inclusive FL in sensitive domains like healthcare. Overall, the work offers a robust path toward regulatory-compliant, trustworthy federated analytics through fixed-memory privacy accounting and verifiable model integrity.

Abstract

Federated learning (FL) enables organizations to collaboratively train models without sharing their datasets. Despite this advantage, recent studies show that both client updates and the global model can leak private information, limiting adoption in sensitive domains such as healthcare. Local differential privacy (LDP) offers strong protection by letting each participant privatize updates before transmission. However, existing LDP methods were designed for centralized training and introduce challenges in FL, including high resource demands that can cause client dropouts and the lack of reliable privacy guarantees under asynchronous participation. These issues undermine model generalizability, fairness, and compliance with regulations such as HIPAA and GDPR. To address them, we propose L-RDP, a DP method designed for LDP that ensures constant, lower memory usage to reduce dropouts and provides rigorous per-client privacy guarantees by accounting for intermittent participation.

Local Differential Privacy for Federated Learning with Fixed Memory Usage and Per-Client Privacy

TL;DR

The paper addresses privacy risks in federated learning, where client updates and the global model can leak information. It introduces L-RDP, a fixed-memory local-DP accountant for FL that uses fixed-size minibatches to stabilize memory usage and provides per-client privacy guarantees under asynchronous participation, complemented by MPC-based model verification to protect model integrity. The approach is implemented in the Flower FL framework, with open-source artifacts forthcoming. Experiments demonstrate substantial memory savings (from up to 46 GB to about 14 GB) with negligible utility loss (around 1% relative drop) across image and NLP tasks at various privacy budgets, indicating practical viability for privacy-preserving, inclusive FL in sensitive domains like healthcare. Overall, the work offers a robust path toward regulatory-compliant, trustworthy federated analytics through fixed-memory privacy accounting and verifiable model integrity.

Abstract

Federated learning (FL) enables organizations to collaboratively train models without sharing their datasets. Despite this advantage, recent studies show that both client updates and the global model can leak private information, limiting adoption in sensitive domains such as healthcare. Local differential privacy (LDP) offers strong protection by letting each participant privatize updates before transmission. However, existing LDP methods were designed for centralized training and introduce challenges in FL, including high resource demands that can cause client dropouts and the lack of reliable privacy guarantees under asynchronous participation. These issues undermine model generalizability, fairness, and compliance with regulations such as HIPAA and GDPR. To address them, we propose L-RDP, a DP method designed for LDP that ensures constant, lower memory usage to reduce dropouts and provides rigorous per-client privacy guarantees by accounting for intermittent participation.
Paper Structure (10 sections, 3 theorems, 19 equations, 3 figures)

This paper contains 10 sections, 3 theorems, 19 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Differential Privacy (DP)
  4. Federated Learning (FL) and Local Differential Privacy (LDP)
  5. Our Proposed LDP Accountant (L-RDP)
  6. Evaluations
  7. Memory Usage
  8. Model Utility
  9. Conclusion and Managerial Implications
  10. Rényi Divergence Upper Bound

Key Result

Lemma 1

For all $j^\prime\in J$, $n\in\mathbb{Z}^+$, there is a measurable map $H_{j^\prime,n}$ such that for all $D$. Moreover, for all $j^\prime$, $n$, the random variables $(B_{j^\prime,n},Z_{j^\prime,n})$ and are independent.

Figures (3)

  • Figure 1: Memory Usage per Training Iteration
  • Figure 2: Accuracy vs. Privacy Budget ($\epsilon$) for Image Recognition
  • Figure 3: Accuracy vs. Privacy Budget ($\epsilon$) for Natural Language Understanding

Theorems & Definitions (7)

  • Definition 1: Differential Privacy dwork2006our
  • Lemma 1
  • Remark 1
  • proof
  • Remark 2
  • Theorem 1
  • Theorem 2