Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Learning with Enhanced Privacy via Model Splitting and Random Client Participation

Yiwei Li, Shuai Wang, Zhuojun Tian, Xiuhua Wang, Shijian Su

TL;DR

MS-PAFL addresses the core challenge of maintaining model utility under differential privacy in Federated Learning by combining model splitting with privacy amplification from random client participation and local data subsampling. By partitioning each client’s model into a private submodel (on-device) and a public submodel (shared for aggregation), and injecting noise only into the public submodel, the framework achieves stronger privacy with reduced utility loss. The authors provide formal single-round and total privacy guarantees, demonstrating that joint amplification tightens privacy bounds beyond what either sampling method achieves alone, and they validate these claims with experiments on the Adult dataset showing improved privacy–utility trade-offs relative to standard DP-FedAvg. The work offers a practical, theoretically grounded approach for privacy-preserving FL that supports flexible participation and scalable deployment while maintaining high model performance under stringent privacy budgets.

Abstract

Federated Learning (FL) often adopts differential privacy (DP) to protect client data, but the added noise required for privacy guarantees can substantially degrade model accuracy. To resolve this challenge, we propose model-splitting privacy-amplified federated learning (MS-PAFL), a novel framework that combines structural model splitting with statistical privacy amplification. In this framework, each client's model is partitioned into a private submodel, retained locally, and a public submodel, shared for global aggregation. The calibrated Gaussian noise is injected only into the public submodel, thereby confining its adverse impact while preserving the utility of the local model. We further present a rigorous theoretical analysis that characterizes the joint privacy amplification achieved through random client participation and local data subsampling under this architecture. The analysis provides tight bounds on both single-round and total privacy loss, demonstrating that MS-PAFL significantly reduces the noise necessary to satisfy a target privacy protection level. Extensive experiments validate our theoretical findings, showing that MS-PAFL consistently attains a superior privacy-utility trade-off and enables the training of highly accurate models under strong privacy guarantees.

Federated Learning with Enhanced Privacy via Model Splitting and Random Client Participation

TL;DR

MS-PAFL addresses the core challenge of maintaining model utility under differential privacy in Federated Learning by combining model splitting with privacy amplification from random client participation and local data subsampling. By partitioning each client’s model into a private submodel (on-device) and a public submodel (shared for aggregation), and injecting noise only into the public submodel, the framework achieves stronger privacy with reduced utility loss. The authors provide formal single-round and total privacy guarantees, demonstrating that joint amplification tightens privacy bounds beyond what either sampling method achieves alone, and they validate these claims with experiments on the Adult dataset showing improved privacy–utility trade-offs relative to standard DP-FedAvg. The work offers a practical, theoretically grounded approach for privacy-preserving FL that supports flexible participation and scalable deployment while maintaining high model performance under stringent privacy budgets.

Abstract

Federated Learning (FL) often adopts differential privacy (DP) to protect client data, but the added noise required for privacy guarantees can substantially degrade model accuracy. To resolve this challenge, we propose model-splitting privacy-amplified federated learning (MS-PAFL), a novel framework that combines structural model splitting with statistical privacy amplification. In this framework, each client's model is partitioned into a private submodel, retained locally, and a public submodel, shared for global aggregation. The calibrated Gaussian noise is injected only into the public submodel, thereby confining its adverse impact while preserving the utility of the local model. We further present a rigorous theoretical analysis that characterizes the joint privacy amplification achieved through random client participation and local data subsampling under this architecture. The analysis provides tight bounds on both single-round and total privacy loss, demonstrating that MS-PAFL significantly reduces the noise necessary to satisfy a target privacy protection level. Extensive experiments validate our theoretical findings, showing that MS-PAFL consistently attains a superior privacy-utility trade-off and enables the training of highly accurate models under strong privacy guarantees.

Paper Structure

This paper contains 33 sections, 8 theorems, 41 equations, 6 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Differentially Private Federated Learning
  4. Privacy Amplification via Sampling
  5. Structural Privacy Mechanisms and Model Splitting
  6. Contributions
  7. Preliminaries
  8. Preliminaries of FL
  9. Preliminaries of DP
  10. Privacy Amplification via Data Subsampling
  11. MS-PAFL: Model-Splitting Privacy-Amplified Federated Learning
  12. The Proposed Random Client Check-in Scheme
  13. The Proposed Privacy-preserving Scheme: MS-PAFL
  14. Model Splitting and Local Updates
  15. Privacy-Preserving Upload
  16. ...and 18 more sections

Key Result

Lemma 1

dwork2014algorithmic Suppose a query function $g$ accesses the dataset $\mathcal{D}_i, \forall i \in [N]$ via randomized mechanism $\mathcal{M}$. Let $\xi$ be zero-mean Gaussian noise with variance $\sigma^2$. Then, the minimal $\sigma^2$ required for $g+ \xi$ to satisfy $(\epsilon_{\bm {\ell}},\del where $s$ is the $\ell_2$-norm sensitivity of the function $g$ defined by

Figures (6)

  • Figure 1: FL system with random client check-in scheme.
  • Figure 2: Comparison of different client participation levels ($p_i \in \{0.1, 0.3, 0.7\}$) under (a) data subsampling WOR and (b) data subsampling WR.
  • Figure 3: Privacy protection level of global model $\epsilon_{c}^t$ under $\epsilon_{{\bm {\ell}}} \in [0,1]$ when (a) $p_i \in \{0.1,0.5,1\}$ and (b) $q_i \in \{0.1, 0.3, 0.5 \}$.
  • Figure 4: Comparison of total privacy loss $\overline{\epsilon}_{c}^T$ under W.O.R. and (a) various values of $p_i$ when $q_i=0.2$, (b) different values of $q_i$ when $p_i=0.5$.
  • Figure 5: Comparison of $\overline{\epsilon}_{i}^T$ under (a) $\epsilon_{\ell} = 0.1$ and (b) $\epsilon_{\ell} = 1$.
  • ...and 1 more figures

Theorems & Definitions (13)

  • Definition 1
  • Lemma 1
  • Definition 2
  • Theorem 1
  • Theorem 2
  • Remark 1
  • Corollary 1
  • Theorem 3
  • Remark 2
  • Theorem 4
  • ...and 3 more