Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ALI-DPFL: Differentially Private Federated Learning with Adaptive Local Iterations

Xinpeng Ling, Jie Fu, Kuncan Wang, Haitao Liu, Zhili Chen

TL;DR

The work tackles privacy-preserving federated learning when both the privacy budget and the number of communication rounds are constrained. It derives a convergence bound $h( au)$ that depends on the number of local DPSGD iterations $ au$ and uses this to compute an adaptive $ au^*$ each round, forming the ALI-DPFL algorithm. Empirical results on MNIST, Fashion-MNIST, and CIFAR-10 show that ALI-DPFL outperforms fixed-τ FedAvg+DP, PE-DPFL, and Adap DP-FL across IID and non-IID data, especially under stringent resource constraints. Privacy guarantees are established via Rényi DP with composition, ensuring DP under the adaptive scheme. The approach offers practical benefits for resource-limited FL deployments by achieving faster convergence and higher accuracy while maintaining formal privacy protections, with code available at the provided GitHub link.

Abstract

Federated Learning (FL) is a distributed machine learning technique that allows model training among multiple devices or organizations by sharing training parameters instead of raw data. However, adversaries can still infer individual information through inference attacks (e.g. differential attacks) on these training parameters. As a result, Differential Privacy (DP) has been widely used in FL to prevent such attacks. We consider differentially private federated learning in a resource-constrained scenario, where both privacy budget and communication rounds are constrained. By theoretically analyzing the convergence, we can find the optimal number of local DPSGD iterations for clients between any two sequential global updates. Based on this, we design an algorithm of Differentially Private Federated Learning with Adaptive Local Iterations (ALI-DPFL). We experiment our algorithm on the MNIST, FashionMNIST and Cifar10 datasets, and demonstrate significantly better performances than previous work in the resource-constraint scenario. Code is available at https://github.com/cheng-t/ALI-DPFL.

ALI-DPFL: Differentially Private Federated Learning with Adaptive Local Iterations

TL;DR

The work tackles privacy-preserving federated learning when both the privacy budget and the number of communication rounds are constrained. It derives a convergence bound that depends on the number of local DPSGD iterations and uses this to compute an adaptive each round, forming the ALI-DPFL algorithm. Empirical results on MNIST, Fashion-MNIST, and CIFAR-10 show that ALI-DPFL outperforms fixed-τ FedAvg+DP, PE-DPFL, and Adap DP-FL across IID and non-IID data, especially under stringent resource constraints. Privacy guarantees are established via Rényi DP with composition, ensuring DP under the adaptive scheme. The approach offers practical benefits for resource-limited FL deployments by achieving faster convergence and higher accuracy while maintaining formal privacy protections, with code available at the provided GitHub link.

Abstract

Federated Learning (FL) is a distributed machine learning technique that allows model training among multiple devices or organizations by sharing training parameters instead of raw data. However, adversaries can still infer individual information through inference attacks (e.g. differential attacks) on these training parameters. As a result, Differential Privacy (DP) has been widely used in FL to prevent such attacks. We consider differentially private federated learning in a resource-constrained scenario, where both privacy budget and communication rounds are constrained. By theoretically analyzing the convergence, we can find the optimal number of local DPSGD iterations for clients between any two sequential global updates. Based on this, we design an algorithm of Differentially Private Federated Learning with Adaptive Local Iterations (ALI-DPFL). We experiment our algorithm on the MNIST, FashionMNIST and Cifar10 datasets, and demonstrate significantly better performances than previous work in the resource-constraint scenario. Code is available at https://github.com/cheng-t/ALI-DPFL.
Paper Structure (26 sections, 6 theorems, 39 equations, 8 figures, 5 tables, 3 algorithms)

This paper contains 26 sections, 6 theorems, 39 equations, 8 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. DPFL
  4. Adaptive local iterations in FL
  5. Preliminaries
  6. Differential privacy
  7. Differentially private stochastic gradient descent
  8. Federated Learning with Differential Privacy
  9. ALI-DPFL Algorithm
  10. Motivation: Convergence Analysis of DPFL
  11. Overview
  12. Our Algorithm
  13. Privacy Analysis
  14. EXPERIMENT
  15. setup
  16. ...and 11 more sections

Key Result

Theorem 1

Given the aforementioned assumptions, we derive the convergence upper bound for Algorithm alg:DPFL as follows: where: Here $\frac{1}{\hat{B}} \triangleq \max_{i,t} \mathbb{E}\frac{1}{|\mathcal{B}^i_t|}$, $\Delta_1=\mathbb{E}\| \bar{\mathbf{w}}_1 - \mathbf{w}^* \|^2$. $\Gamma=F^* - \sum_{i=1}^{N} p_i F_i^*$, which is consistent with Li2019OnTheConvergence, where a larger value of $\Gamma$ indicat

Figures (8)

  • Figure 1: Iters=150
  • Figure 2: Iters=300
  • Figure 3: Iters=600
  • Figure 5: The Framework of Differentially Private Federated Learning with Adaptive Local Iterations
  • Figure 6: Workflow of ALI-DPFL
  • ...and 3 more figures

Theorems & Definitions (10)

  • Definition 1
  • Theorem 1
  • Lemma 1
  • Lemma 2
  • Theorem 2
  • Definition 2
  • Definition 3
  • Lemma 3
  • Definition 4
  • Lemma 4