Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PAC-DP: Personalized Adaptive Clipping for Differentially Private Federated Learning

Hao Zhou, Siqi Cai, Hua Dai, Geng Yang, Jing Luo, Hui Cai

Abstract

Differential privacy (DP) is crucial for safeguarding sensitive client information in federated learning (FL), yet traditional DP-FL methods rely predominantly on fixed gradient clipping thresholds. Such static clipping neglects significant client heterogeneity and varying privacy sensitivities, which may lead to an unfavorable privacy-utility trade-off. In this paper, we propose PAC-DP, a Personalized Adaptive Clipping framework for federated learning under record-level local differential privacy. PAC-DP introduces a Simulation-CurveFitting approach leveraging a server-hosted public proxy dataset to learn an effective mapping between personalized privacy budgets epsilon and gradient clipping thresholds C, which is then deployed online with a lightweight round-wise schedule. This design enables budget-conditioned threshold selection while avoiding data-dependent tuning during training. We provide theoretical analyses establishing convergence guarantees under the per-example clipping and Gaussian perturbation mechanism and a reproducible privacy accounting procedure. Extensive evaluations on multiple FL benchmarks show that PAC-DP surpasses conventional fixed-threshold approaches under matched privacy budgets, improving accuracy by up to 26% and accelerating convergence by up to 45.5% in our evaluated settings.

PAC-DP: Personalized Adaptive Clipping for Differentially Private Federated Learning

Abstract

Differential privacy (DP) is crucial for safeguarding sensitive client information in federated learning (FL), yet traditional DP-FL methods rely predominantly on fixed gradient clipping thresholds. Such static clipping neglects significant client heterogeneity and varying privacy sensitivities, which may lead to an unfavorable privacy-utility trade-off. In this paper, we propose PAC-DP, a Personalized Adaptive Clipping framework for federated learning under record-level local differential privacy. PAC-DP introduces a Simulation-CurveFitting approach leveraging a server-hosted public proxy dataset to learn an effective mapping between personalized privacy budgets epsilon and gradient clipping thresholds C, which is then deployed online with a lightweight round-wise schedule. This design enables budget-conditioned threshold selection while avoiding data-dependent tuning during training. We provide theoretical analyses establishing convergence guarantees under the per-example clipping and Gaussian perturbation mechanism and a reproducible privacy accounting procedure. Extensive evaluations on multiple FL benchmarks show that PAC-DP surpasses conventional fixed-threshold approaches under matched privacy budgets, improving accuracy by up to 26% and accelerating convergence by up to 45.5% in our evaluated settings.
Paper Structure (25 sections, 8 theorems, 59 equations, 13 figures, 2 tables, 2 algorithms)

This paper contains 25 sections, 8 theorems, 59 equations, 13 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Problem Formulation
  5. Federated Learning with Adaptive Gradient Clipping Personalized DP
  6. Challenges in Personalized DP Clipping
  7. Solution Overview
  8. Round-wise Schedule for Clipping
  9. Schedule design.
  10. Adaptive Personalized DP Training Algorithm
  11. Privacy and Utility Analysis
  12. Privacy Objectives
  13. Utility Objectives
  14. Bias--variance view of choosing $C(\varepsilon)$.
  15. Selecting Clipping Thresholds via Simulation-CurveFitting
  16. ...and 10 more sections

Key Result

Theorem 1

In round $t$, for client $i$, the mechanism in Eq. eq:dp_noise_multiplier satisfies $(\varepsilon_i^t,\delta)$-local DP under record-level adjacency, where

Figures (13)

  • Figure 1: Personalized Adaptive Clipping Mechanism
  • Figure 2: Overview of the PAC-DP framework. A proxy dataset is used to simulate privacy-utility trade-offs and fit a mapping $C^* = F(\varepsilon)$, which guides personalized clipping and noise injection during training.
  • Figure 3: Relationship between Privacy Budget $\varepsilon$ and Optimal Clipping Threshold $C_{\max}$ on MNIST and CIFAR-10 Datasets. Despite dataset complexity differences, the fitted mapping remains stable and generalizable.
  • Figure 4: Performance Comparison on MNIST under Various Privacy Budgets. PAC-DP achieves consistent accuracy gains and faster convergence across both fixed and adaptive baselines.
  • Figure 5: Performance Comparison on CIFAR-10 under Fixed Thresholds and PAC-DP. Despite higher data complexity, PAC-DP maintains robust convergence and superior accuracy.
  • ...and 8 more figures

Theorems & Definitions (16)

  • Definition 1: $(\varepsilon, \delta)$-DP
  • Definition 2: Record-level Local DP
  • Theorem 1: Per-round Record-level Local DP
  • proof
  • Corollary 1: Per-client Composition over Participation Rounds
  • Lemma 1: Per-round RDP of the Gaussian mechanism
  • Theorem 2: Non-convex Convergence
  • proof
  • Theorem 3: Convex Convergence
  • proof
  • ...and 6 more