Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Training Differentially Private Models with Secure Multiparty Computation

Sikha Pentyala, Davis Railsback, Ricardo Maia, Rafael Dowsley, David Melanson, Anderson Nascimento, Martine De Cock

TL;DR

This work tackles private learning from data held by multiple data owners by marrying secure multiparty computation (MPC) with differential privacy (DP). The approach trains a logistic regression model via MPC and privately adds DP noise to the secret-shared coefficients using an output-perturbation mechanism, with noise drawn from a distribution proportional to $e^{-\frac{n\epsilon\Lambda}{2}\|\eta\|}$, ensuring $$(\epsilon,0)$$-DP while preserving input privacy. It supports both horizontally and vertically distributed data, avoids a central trusted curator, and demonstrates superior utility to pure DP baselines in empirical tests, including iDASH 2021 Track III and other biomedical datasets. The results indicate practical viability with honest-majority MPC settings and scalability across multiple computing parties, highlighting a path toward wider deployment of privacy-preserving collaborative learning in sensitive domains.

Abstract

We address the problem of learning a machine learning model from training data that originates at multiple data owners while providing formal privacy guarantees regarding the protection of each owner's data. Existing solutions based on Differential Privacy (DP) achieve this at the cost of a drop in accuracy. Solutions based on Secure Multiparty Computation (MPC) do not incur such accuracy loss but leak information when the trained model is made publicly available. We propose an MPC solution for training DP models. Our solution relies on an MPC protocol for model training, and an MPC protocol for perturbing the trained model coefficients with Laplace noise in a privacy-preserving manner. The resulting MPC+DP approach achieves higher accuracy than a pure DP approach while providing the same formal privacy guarantees. Our work obtained first place in the iDASH2021 Track III competition on confidential computing for secure genome analysis.

Training Differentially Private Models with Secure Multiparty Computation

TL;DR

This work tackles private learning from data held by multiple data owners by marrying secure multiparty computation (MPC) with differential privacy (DP). The approach trains a logistic regression model via MPC and privately adds DP noise to the secret-shared coefficients using an output-perturbation mechanism, with noise drawn from a distribution proportional to , ensuring -DP while preserving input privacy. It supports both horizontally and vertically distributed data, avoids a central trusted curator, and demonstrates superior utility to pure DP baselines in empirical tests, including iDASH 2021 Track III and other biomedical datasets. The results indicate practical viability with honest-majority MPC settings and scalability across multiple computing parties, highlighting a path toward wider deployment of privacy-preserving collaborative learning in sensitive domains.

Abstract

We address the problem of learning a machine learning model from training data that originates at multiple data owners while providing formal privacy guarantees regarding the protection of each owner's data. Existing solutions based on Differential Privacy (DP) achieve this at the cost of a drop in accuracy. Solutions based on Secure Multiparty Computation (MPC) do not incur such accuracy loss but leak information when the trained model is made publicly available. We propose an MPC solution for training DP models. Our solution relies on an MPC protocol for model training, and an MPC protocol for perturbing the trained model coefficients with Laplace noise in a privacy-preserving manner. The resulting MPC+DP approach achieves higher accuracy than a pure DP approach while providing the same formal privacy guarantees. Our work obtained first place in the iDASH2021 Track III competition on confidential computing for secure genome analysis.
Paper Structure (15 sections, 2 equations, 1 figure, 8 tables, 4 algorithms)

This paper contains 15 sections, 2 equations, 1 figure, 8 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Related Work
  4. Method
  5. Results
  6. Conclusion
  7. Pseudocode
  8. Additional experiments
  9. Effect of Privacy Budget $\epsilon$ on Accuracy of Models Trained with $\pi_{\mathsf{LR}}$+$\pi_{\mathsf{DP}}$
  10. Comparison with Other Perturbation Techniques
  11. Comparison with Other Methods on Horizontally Partitioned Data
  12. Experiments on Other Data Sets
  13. GSE2034
  14. BC-TCGA
  15. Scalability of $\pi_{\mathsf{LR}}\xspace$+$\pi_{\mathsf{DP}}\xspace$ with Number of Computing Parties

Figures (1)

  • Figure 1: Privacy-preserving training of an $\epsilon$-DP model with MPC