Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mitigating Privacy-Utility Trade-off in Decentralized Federated Learning via $f$-Differential Privacy

Xiang Li, Buxin Su, Chendi Wang, Qi Long, Weijie J. Su

TL;DR

The paper develops a fine-grained privacy accounting framework for decentralized Federated Learning by introducing two $f$-differential privacy notions: Pairwise Network $f$-DP (PN-$f$-DP) for random-walk communication and Secret-based $f$-Local DP (Sec-$f$-LDP) for correlated-noise, secret-sharing scenarios. It analyzes two practical DP-SGD variants—random-walk decentralized DP-SGD and DecoR-style correlated-noise DP-SGD—deriving user- and record-level privacy guarantees via mixture distributions and Markov-chain concentration. The framework yields tighter $(oldsymbol{ ext{e}}, oldsymbol{ ext{d}})$-DP bounds than Rényi DP baselines and demonstrates improved utility on synthetic graphs and real datasets (housing and MNIST). This work enhances privacy-utility trade-offs in decentralized FL and provides a pathway to robust, implementable privacy accounting under sparse, iterative, and potentially colluding network conditions.

Abstract

Differentially private (DP) decentralized Federated Learning (FL) allows local users to collaborate without sharing their data with a central server. However, accurately quantifying the privacy budget of private FL algorithms is challenging due to the co-existence of complex algorithmic components such as decentralized communication and local updates. This paper addresses privacy accounting for two decentralized FL algorithms within the $f$-differential privacy ($f$-DP) framework. We develop two new $f$-DP-based accounting methods tailored to decentralized settings: Pairwise Network $f$-DP (PN-$f$-DP), which quantifies privacy leakage between user pairs under random-walk communication, and Secret-based $f$-Local DP (Sec-$f$-LDP), which supports structured noise injection via shared secrets. By combining tools from $f$-DP theory and Markov chain concentration, our accounting framework captures privacy amplification arising from sparse communication, local iterations, and correlated noise. Experiments on synthetic and real datasets demonstrate that our methods yield consistently tighter $(ε,δ)$ bounds and improved utility compared to Rényi DP-based approaches, illustrating the benefits of $f$-DP in decentralized privacy accounting.

Mitigating Privacy-Utility Trade-off in Decentralized Federated Learning via $f$-Differential Privacy

TL;DR

The paper develops a fine-grained privacy accounting framework for decentralized Federated Learning by introducing two -differential privacy notions: Pairwise Network -DP (PN--DP) for random-walk communication and Secret-based -Local DP (Sec--LDP) for correlated-noise, secret-sharing scenarios. It analyzes two practical DP-SGD variants—random-walk decentralized DP-SGD and DecoR-style correlated-noise DP-SGD—deriving user- and record-level privacy guarantees via mixture distributions and Markov-chain concentration. The framework yields tighter -DP bounds than Rényi DP baselines and demonstrates improved utility on synthetic graphs and real datasets (housing and MNIST). This work enhances privacy-utility trade-offs in decentralized FL and provides a pathway to robust, implementable privacy accounting under sparse, iterative, and potentially colluding network conditions.

Abstract

Differentially private (DP) decentralized Federated Learning (FL) allows local users to collaborate without sharing their data with a central server. However, accurately quantifying the privacy budget of private FL algorithms is challenging due to the co-existence of complex algorithmic components such as decentralized communication and local updates. This paper addresses privacy accounting for two decentralized FL algorithms within the -differential privacy (-DP) framework. We develop two new -DP-based accounting methods tailored to decentralized settings: Pairwise Network -DP (PN--DP), which quantifies privacy leakage between user pairs under random-walk communication, and Secret-based -Local DP (Sec--LDP), which supports structured noise injection via shared secrets. By combining tools from -DP theory and Markov chain concentration, our accounting framework captures privacy amplification arising from sparse communication, local iterations, and correlated noise. Experiments on synthetic and real datasets demonstrate that our methods yield consistently tighter bounds and improved utility compared to Rényi DP-based approaches, illustrating the benefits of -DP in decentralized privacy accounting.
Paper Structure (57 sections, 22 theorems, 45 equations, 11 figures, 4 tables, 2 algorithms)

This paper contains 57 sections, 22 theorems, 45 equations, 11 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contribution.
  3. Related works
  4. DP notions in FL.
  5. Privacy amplification in FL.
  6. Preliminaries
  7. Preliminaries on differential privacy
  8. Pairwise network differential privacy
  9. Communication graph.
  10. Two DP levels.
  11. Pairwise network differential privacy (PN-DP).
  12. -Differential Privacy Notions for Decentralized Learning
  13. Privacy Analysis
  14. User-level Privacy Analysis
  15. Mixture distributions from random walks.
  16. ...and 42 more sections

Key Result

Lemma 4.1

For any $D\sim_i D'$, we have $T(\mathcal{A}_j^{\mathrm{single}}(D),\mathcal{A}_j^{\mathrm{single}}(D'))\geq f_{ij}^{\mathrm{single}},$ where $f_{ij}^{\mathrm{single}}$ is defined as follows: for all $s \in [0, \infty)$, with $\alpha_t(s) = \Pr_{X\sim P_j^t}\left[\frac{p_j^t(X)}{q_j^t(X)} > s \right]$ for any $t$, $\alpha_{T+1}(s) = \mathbf{1}_{[s<1]},$ and $\alpha(s) = \sum_{t=1}^{T+1} w^t_{ij}\

Figures (11)

  • Figure 1: Comparisons by converting PN-$f$-DP and PN-RDP to PN-$(\epsilon, \delta)$-DP on Hypercube. See more experimental results in Appendix \ref{['appe:RDP']}.
  • Figure 2: Top: Performance of private logistic regression on the Housing dataset. Bottom: Performance of private MNIST classification using a neural network.
  • Figure 3: Left: Privacy-utility trade-offs for our $f$-DP, DECOR, and LDP when converted to the same $(\epsilon, 10^{-5})$-DP. Right: Optimality gap vs. iteration steps for Alg. \ref{['alg:decor']} using different privacy accounting methods with $\epsilon = 3$. Similar results for other values of $\epsilon \in \{5, 7, 10\}$ are provided in Appendix \ref{['appen:correlated']}.
  • Figure 4: Visualization of synthetic graphs.
  • Figure 5: Comparison of the numerical conversion to $(\epsilon, \delta)$-DP on the clique (top) and regular (bottom) graphs.
  • ...and 6 more figures

Theorems & Definitions (54)

  • Definition 2.1: $(\epsilon,\delta)$-DP
  • Definition 2.2: Rényi DP DBLP:conf/csfw/Mironov17
  • Definition 2.3: $f$-DP and Gaussian DP dong2019gaussian
  • Definition 2.4: Tensor product of trade-off functions dong2019gaussian
  • Definition 2.5: User-level pairwise network RDP, or PN-RDP DBLP:conf/nips/CyffersEBM22
  • Definition 3.1: User-level PN-$f$-DP
  • Definition 3.2: Record-level PN-$f$-DP
  • Remark 3.1
  • Definition 3.3: Sec-$f$-LDP
  • Remark 4.1
  • ...and 44 more