Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Privacy in Federated Learning through Local Training

Nicola Bastianello, Changxin Liu, Karl H. Johansson

TL;DR

This paper tackles privacy and communication efficiency in federated learning by proposing Fed-PLT, a PRS-based algorithm that enables partial participation and local training with flexible local solvers. It provides rigorous convergence analysis showing exact convergence under strong convexity with gradient-based local solvers and convergence to a neighborhood when stochastic/noisy solvers are used, while also deriving differential privacy bounds that depend on the number of local epochs and gradient noise. The approach unifies privacy guarantees with communication efficiency and composite-cost optimization, and is validated through numerical experiments on logistic regression, illustrating favorable comparisons to state-of-the-art methods and highlighting practical trade-offs between privacy, accuracy, and communication. The practical impact lies in offering a modular, privacy-preserving federated framework that reduces communications without sacrificing accuracy, while providing explicit privacy-utility guidelines for selecting local-epoch counts and noise levels.

Abstract

In this paper we propose the federated learning algorithm Fed-PLT to overcome the challenges of (i) expensive communications and (ii) privacy preservation. We address (i) by allowing for both partial participation and local training, which significantly reduce the number of communication rounds between the central coordinator and computing agents. The algorithm matches the state of the art in the sense that the use of local training demonstrably does not impact accuracy. Additionally, agents have the flexibility to choose from various local training solvers, such as (stochastic) gradient descent and accelerated gradient descent. Further, we investigate how employing local training can enhance privacy, addressing point (ii). In particular, we derive differential privacy bounds and highlight their dependence on the number of local training epochs. We assess the effectiveness of the proposed algorithm by comparing it to alternative techniques, considering both theoretical analysis and numerical results from a classification task.

Enhancing Privacy in Federated Learning through Local Training

TL;DR

This paper tackles privacy and communication efficiency in federated learning by proposing Fed-PLT, a PRS-based algorithm that enables partial participation and local training with flexible local solvers. It provides rigorous convergence analysis showing exact convergence under strong convexity with gradient-based local solvers and convergence to a neighborhood when stochastic/noisy solvers are used, while also deriving differential privacy bounds that depend on the number of local epochs and gradient noise. The approach unifies privacy guarantees with communication efficiency and composite-cost optimization, and is validated through numerical experiments on logistic regression, illustrating favorable comparisons to state-of-the-art methods and highlighting practical trade-offs between privacy, accuracy, and communication. The practical impact lies in offering a modular, privacy-preserving federated framework that reduces communications without sacrificing accuracy, while providing explicit privacy-utility guidelines for selecting local-epoch counts and noise levels.

Abstract

In this paper we propose the federated learning algorithm Fed-PLT to overcome the challenges of (i) expensive communications and (ii) privacy preservation. We address (i) by allowing for both partial participation and local training, which significantly reduce the number of communication rounds between the central coordinator and computing agents. The algorithm matches the state of the art in the sense that the use of local training demonstrably does not impact accuracy. Additionally, agents have the flexibility to choose from various local training solvers, such as (stochastic) gradient descent and accelerated gradient descent. Further, we investigate how employing local training can enhance privacy, addressing point (ii). In particular, we derive differential privacy bounds and highlight their dependence on the number of local training epochs. We assess the effectiveness of the proposed algorithm by comparing it to alternative techniques, considering both theoretical analysis and numerical results from a classification task.
Paper Structure (44 sections, 13 theorems, 57 equations, 1 figure, 9 tables, 1 algorithm)

This paper contains 44 sections, 13 theorems, 57 equations, 1 figure, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Preliminaries
  4. Operator theory
  5. Application to convex optimization
  6. Stochastic operator theory
  7. Differential privacy
  8. Problem Formulation
  9. Problem
  10. Design objectives
  11. Communication burden
  12. Heterogeneous agents
  13. Privacy
  14. Algorithm Design
  15. Algorithm derivation
  16. ...and 29 more sections

Key Result

Lemma 1

Let $\mathcal{T} : \mathbb{R}^n \to \mathbb{R}^n$ be $\zeta$-contractive; then $\mathcal{T}$ has a unique fixed point $\bar{\mathbold{x}}$, which is the limit of the sequence generated by: In particular, it holds $\left\lVert\mathbold{x}^\ell - \bar{\mathbold{x}}\right\rVert \leq \zeta^\ell \left\lVert\mathbold{x}^0 - \bar{\mathbold{x}}\right\rVert.$

Figures (1)

  • Figure 1: The federated architecture.

Theorems & Definitions (27)

  • Definition 1: Fixed points
  • Definition 2: Contractive operators
  • Lemma 1: Banach-Picard
  • Lemma 2: Gradient descent
  • Definition 3: Proximal, reflective operators
  • Lemma 3: Peaceman-Rachford splitting
  • Lemma 4: Stochastic Banach-Picard
  • Definition 4: Rényi differential privacy
  • Definition 5: Approximate differential privacy
  • Lemma 5: RDP to ADP conversion
  • ...and 17 more