Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ULDP-FL: Federated Learning with Across Silo User-Level Differential Privacy

Fumiyuki Kato, Li Xiong, Shun Takagi, Yang Cao, Masatoshi Yoshikawa

TL;DR

This work defines and addresses user-level differential privacy (ULDP) in cross-silo federated learning, where a single user can have records across multiple silos. It introduces ULDP-AVG/SGD with per-user weighted clipping to directly bound user-level sensitivity, avoiding the noisy overhead of group-DP conversions. A private weighting protocol combines secure aggregation, Paillier encryption, and multiplicative blinding to compute privately optimized per-user weights, supporting an enhanced weighting strategy that improves utility under ULDP. Theoretical analyses establish privacy and convergence guarantees for ULDP-AVG, and extensive experiments on real-world datasets demonstrate favorable privacy-utility trade-offs compared with baselines, including the practicality of the private protocol. Overall, the paper delivers the first cross-silo ULDP FL framework with practical privacy protection for users spanning multiple silos and provides a pathway toward scalable private implementations.

Abstract

Differentially Private Federated Learning (DP-FL) has garnered attention as a collaborative machine learning approach that ensures formal privacy. Most DP-FL approaches ensure DP at the record-level within each silo for cross-silo FL. However, a single user's data may extend across multiple silos, and the desired user-level DP guarantee for such a setting remains unknown. In this study, we present Uldp-FL, a novel FL framework designed to guarantee user-level DP in cross-silo FL where a single user's data may belong to multiple silos. Our proposed algorithm directly ensures user-level DP through per-user weighted clipping, departing from group-privacy approaches. We provide a theoretical analysis of the algorithm's privacy and utility. Additionally, we enhance the utility of the proposed algorithm with an enhanced weighting strategy based on user record distribution and design a novel private protocol that ensures no additional information is revealed to the silos and the server. Experiments on real-world datasets show substantial improvements in our methods in privacy-utility trade-offs under user-level DP compared to baseline methods. To the best of our knowledge, our work is the first FL framework that effectively provides user-level DP in the general cross-silo FL setting.

ULDP-FL: Federated Learning with Across Silo User-Level Differential Privacy

TL;DR

This work defines and addresses user-level differential privacy (ULDP) in cross-silo federated learning, where a single user can have records across multiple silos. It introduces ULDP-AVG/SGD with per-user weighted clipping to directly bound user-level sensitivity, avoiding the noisy overhead of group-DP conversions. A private weighting protocol combines secure aggregation, Paillier encryption, and multiplicative blinding to compute privately optimized per-user weights, supporting an enhanced weighting strategy that improves utility under ULDP. Theoretical analyses establish privacy and convergence guarantees for ULDP-AVG, and extensive experiments on real-world datasets demonstrate favorable privacy-utility trade-offs compared with baselines, including the practicality of the private protocol. Overall, the paper delivers the first cross-silo ULDP FL framework with practical privacy protection for users spanning multiple silos and provides a pathway toward scalable private implementations.

Abstract

Differentially Private Federated Learning (DP-FL) has garnered attention as a collaborative machine learning approach that ensures formal privacy. Most DP-FL approaches ensure DP at the record-level within each silo for cross-silo FL. However, a single user's data may extend across multiple silos, and the desired user-level DP guarantee for such a setting remains unknown. In this study, we present Uldp-FL, a novel FL framework designed to guarantee user-level DP in cross-silo FL where a single user's data may belong to multiple silos. Our proposed algorithm directly ensures user-level DP through per-user weighted clipping, departing from group-privacy approaches. We provide a theoretical analysis of the algorithm's privacy and utility. Additionally, we enhance the utility of the proposed algorithm with an enhanced weighting strategy based on user record distribution and design a novel private protocol that ensures no additional information is revealed to the silos and the server. Experiments on real-world datasets show substantial improvements in our methods in privacy-utility trade-offs under user-level DP compared to baseline methods. To the best of our knowledge, our work is the first FL framework that effectively provides user-level DP in the general cross-silo FL setting.
Paper Structure (29 sections, 13 theorems, 28 equations, 12 figures, 2 tables, 5 algorithms)

This paper contains 29 sections, 13 theorems, 28 equations, 12 figures, 2 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Background & Preliminaries
  3. Cross-silo Federated learning
  4. Differential Privacy
  5. Differentially Private FL
  6. Uldp-FL Framework
  7. Trust model and Assumptions
  8. Privacy definition
  9. Baseline methods: ULDP-NAIVE/GROUP
  10. Advanced methods: ULDP-AVG/SGD
  11. Enhanced Weighting and Private weighting protocol
  12. The weighting protocol
  13. Theoretical analysis
  14. Experiments
  15. Settings
  16. ...and 14 more sections

Key Result

Lemma 1

If $\mathcal{M}_{1}$ satisfies $(\alpha, \rho_1)$-RDP and $\mathcal{M}_{2}$ satisfies $(\alpha, \rho_2)$, then their composition $\mathcal{M}_{1} \circ \mathcal{M}_{2}$ satisfies $(\alpha, \rho_1 + \rho_2)$-RDP.

Figures (12)

  • Figure 1: In cross-silo FL, records belonging to the same user can exist across silos, e.g., a user can use several credit card companies. In this study, we investigate how to train models satisfying user-level DP in this setting.
  • Figure 2: Group-privacy conversion results.
  • Figure 3: An intuitive illustration of the difference between ULDP-NAIVE and ULDP-AVG. In ULDP-NAIVE, every user can contribute to whole model deltas. In ULDP-AVG, one user's contribution is limited to a small portion, i.e., $1/|U|$ of the whole model delta, which reduces user-level sensitivity.
  • Figure 4: Privacy-utility trade-offs on Creditcard dataset: Test Accuracy (Left), Privacy (Right).
  • Figure 5: Privacy-utility trade-offs on MNIST dataset: Test Loss (Left), Accuracy (Middle), Privacy (Right).
  • ...and 7 more figures

Theorems & Definitions (22)

  • Definition 1: $(\epsilon, \delta)$-DP
  • Definition 2: $(\alpha, \rho)$-RDP mironov2017renyi
  • Lemma 1: RDP composition mironov2017renyi
  • Lemma 2: RDP to DP conversion balle2020hypothesis
  • Lemma 3: RDP Gaussian mechanism mironov2017renyi
  • Lemma 4: RDP for sub-sampled Gaussian mechanism wang2019subsampled
  • Definition 3: $(k,\epsilon, \delta)$-GDP
  • Lemma 5: Group privacy conversion (record-level DP to GDP) gautam2020lec5
  • Lemma 6: Group-privacy of RDP (record-level DP to GDP) mironov2017renyi
  • Definition 4: $(\epsilon, \delta)$-User-Level DP (ULDP)
  • ...and 12 more