Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CorBin-FL: A Differentially Private Federated Learning Mechanism using Common Randomness

Hojat Allah Salehi, Md Jueal Mia, S. Sandeep Pradhan, M. Hadi Amini, Farhad Shirani

TL;DR

This work tackles privacy-utility trade-offs in federated learning by introducing CorBin-FL, a differentially private mechanism that uses correlated binary stochastic quantization with shared randomness to achieve parameter-level PLDP while preserving accuracy. It extends to AugCorBin-FL, which simultaneously delivers user-level and sample-level central DP guarantees, backed by theoretical bounds on privacy parameters and mean-squared error. The approach leverages common randomness exchanged via secure channels between paired clients to generate correlated quantizers, yielding unbiased server updates and improved MSE relative to baselines such as Gaussian, Laplacian, and LDP-FL under equal PLDP budgets. Extensive experiments on MNIST and CIFAR-10 demonstrate higher accuracy with limited communication costs (one-bit quantization) compared to traditional DP mechanisms, highlighting practical impact for privacy-preserving FL in resource-constrained settings.

Abstract

Federated learning (FL) has emerged as a promising framework for distributed machine learning. It enables collaborative learning among multiple clients, utilizing distributed data and computing resources. However, FL faces challenges in balancing privacy guarantees, communication efficiency, and overall model accuracy. In this work, we introduce CorBin-FL, a privacy mechanism that uses correlated binary stochastic quantization to achieve differential privacy while maintaining overall model accuracy. The approach uses secure multi-party computation techniques to enable clients to perform correlated quantization of their local model updates without compromising individual privacy. We provide theoretical analysis showing that CorBin-FL achieves parameter-level local differential privacy (PLDP), and that it asymptotically optimizes the privacy-utility trade-off between the mean square error utility measure and the PLDP privacy measure. We further propose AugCorBin-FL, an extension that, in addition to PLDP, achieves user-level and sample-level central differential privacy guarantees. For both mechanisms, we derive bounds on privacy parameters and mean squared error performance measures. Extensive experiments on MNIST and CIFAR10 datasets demonstrate that our mechanisms outperform existing differentially private FL mechanisms, including Gaussian and Laplacian mechanisms, in terms of model accuracy under equal PLDP privacy budgets.

CorBin-FL: A Differentially Private Federated Learning Mechanism using Common Randomness

TL;DR

This work tackles privacy-utility trade-offs in federated learning by introducing CorBin-FL, a differentially private mechanism that uses correlated binary stochastic quantization with shared randomness to achieve parameter-level PLDP while preserving accuracy. It extends to AugCorBin-FL, which simultaneously delivers user-level and sample-level central DP guarantees, backed by theoretical bounds on privacy parameters and mean-squared error. The approach leverages common randomness exchanged via secure channels between paired clients to generate correlated quantizers, yielding unbiased server updates and improved MSE relative to baselines such as Gaussian, Laplacian, and LDP-FL under equal PLDP budgets. Extensive experiments on MNIST and CIFAR-10 demonstrate higher accuracy with limited communication costs (one-bit quantization) compared to traditional DP mechanisms, highlighting practical impact for privacy-preserving FL in resource-constrained settings.

Abstract

Federated learning (FL) has emerged as a promising framework for distributed machine learning. It enables collaborative learning among multiple clients, utilizing distributed data and computing resources. However, FL faces challenges in balancing privacy guarantees, communication efficiency, and overall model accuracy. In this work, we introduce CorBin-FL, a privacy mechanism that uses correlated binary stochastic quantization to achieve differential privacy while maintaining overall model accuracy. The approach uses secure multi-party computation techniques to enable clients to perform correlated quantization of their local model updates without compromising individual privacy. We provide theoretical analysis showing that CorBin-FL achieves parameter-level local differential privacy (PLDP), and that it asymptotically optimizes the privacy-utility trade-off between the mean square error utility measure and the PLDP privacy measure. We further propose AugCorBin-FL, an extension that, in addition to PLDP, achieves user-level and sample-level central differential privacy guarantees. For both mechanisms, we derive bounds on privacy parameters and mean squared error performance measures. Extensive experiments on MNIST and CIFAR10 datasets demonstrate that our mechanisms outperform existing differentially private FL mechanisms, including Gaussian and Laplacian mechanisms, in terms of model accuracy under equal PLDP privacy budgets.
Paper Structure (37 sections, 5 theorems, 70 equations, 4 figures, 3 algorithms)

This paper contains 37 sections, 5 theorems, 70 equations, 4 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Private Federated Learning
  4. Local and Central Differential Privacy
  5. User-Level Central Differential Privacy:
  6. Sample-Level Central Differential Privacy:
  7. Parameter-Level Local Differential Privacy:
  8. The LDP-FL Mechanism
  9. Correlated Stochastic Quantizers
  10. Optimality of the LDPQ Stochastic Quantizer
  11. Correlated Stochastic Quantization
  12. CorBin-FL and AugCorBin-FL Mechanisms
  13. The CorBin-FL Mechanism:
  14. Privacy-Utility Guarantees:
  15. The Augmented CorBin-FL Mechanism:
  16. ...and 22 more sections

Key Result

Proposition 1

Let $c,r,\epsilon_p>0$. The binary-output quantizer $Q(w)= LDPQ(\epsilon_p,c,r,w), w\in [c-r,c+r]$ in Algorithm Alg:algQ is the unique quantizer satisfying conditions C1-C3.

Figures (4)

  • Figure 1: Comparison of different privacy mechanisms (Experiment 1).
  • Figure 2: Experimental results for (a) number of shared common random bits (Experiment 2), (b) dropout probability (Experiment 3) and (c) number of clients (Experiment 4).
  • Figure 3: Additional experimental results evaluating (a) the accuracy performance of LDP-FL, CorBin-FL, and CQ suresh2022correlated, (b) Performance of LDP-FL, AugCorBin-FL, and CorBin-FL under various PLDP privacy budgets, and (c) accuracy performance of LDP-FL and CorBin-FL on the MNIST dataset.
  • Figure 4: MSE of CorBinQ and LDPQ pairs of quantizers for inputs $w_1,w_2\in[-0.5,0.5]$.

Theorems & Definitions (5)

  • Proposition 1
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4