Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Privacy-Preserving Fully Distributed Gaussian Process Regression

Yeongjun Jang, Kaoru Teranishi, Jihoon Suh, Takashi Tanaka

TL;DR

This work tackles privacy in distributed Gaussian process regression (GPR) by introducing a privacy-preserving fully distributed GPR protocol built on secure multi-party computation and secure average consensus. It guarantees that local datasets remain confidential while the agents' local models converge to the global DGPR solution, without relying on a trusted central server, and it provides simulation-based privacy guarantees under a semi-honest adversary model. A novel extension enables kernel hyperparameter optimization in a privacy-preserving manner, using local log marginal likelihoods and a consensus-based approach. Experimental results on real benchmarks demonstrate the method’s effectiveness, scalability, and practical applicability in privacy-sensitive collaborative learning tasks such as healthcare and finance.

Abstract

Although distributed Gaussian process regression (GPR) enables multiple agents with separate datasets to jointly learn a model of the target function, its collaborative nature poses risks of private data leakage. To address this, we propose a privacy-preserving fully distributed GPR protocol based on secure multi-party computation (SMPC) that preserves the confidentiality of each agent's local dataset. Building upon a secure distributed average consensus algorithm, the protocol guarantees that each agent's local model practically converges to the same global model that would be obtained by the standard distributed GPR. Further, we adopt the paradigm of simulation based security to provide formal privacy guarantees, and extend the proposed protocol to enable kernel hyperparameter optimization, which is critical yet often overlooked in the literature. Experimental results demonstrate the effectiveness and practical applicability of the proposed method.

Privacy-Preserving Fully Distributed Gaussian Process Regression

TL;DR

This work tackles privacy in distributed Gaussian process regression (GPR) by introducing a privacy-preserving fully distributed GPR protocol built on secure multi-party computation and secure average consensus. It guarantees that local datasets remain confidential while the agents' local models converge to the global DGPR solution, without relying on a trusted central server, and it provides simulation-based privacy guarantees under a semi-honest adversary model. A novel extension enables kernel hyperparameter optimization in a privacy-preserving manner, using local log marginal likelihoods and a consensus-based approach. Experimental results on real benchmarks demonstrate the method’s effectiveness, scalability, and practical applicability in privacy-sensitive collaborative learning tasks such as healthcare and finance.

Abstract

Although distributed Gaussian process regression (GPR) enables multiple agents with separate datasets to jointly learn a model of the target function, its collaborative nature poses risks of private data leakage. To address this, we propose a privacy-preserving fully distributed GPR protocol based on secure multi-party computation (SMPC) that preserves the confidentiality of each agent's local dataset. Building upon a secure distributed average consensus algorithm, the protocol guarantees that each agent's local model practically converges to the same global model that would be obtained by the standard distributed GPR. Further, we adopt the paradigm of simulation based security to provide formal privacy guarantees, and extend the proposed protocol to enable kernel hyperparameter optimization, which is critical yet often overlooked in the literature. Experimental results demonstrate the effectiveness and practical applicability of the proposed method.

Paper Structure

This paper contains 27 sections, 6 theorems, 62 equations, 3 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Notations
  4. Preliminaries and Problem Formulation
  5. Distributed Gaussian Process Regression
  6. Problem Formulation
  7. Secure Distributed Average Consensus
  8. Additive Secret Sharing
  9. Average Consensus with Quantized Communication
  10. Protocol Design
  11. Generation of Masking Terms
  12. Privacy Analysis
  13. Privacy-preserving Fully Distributed GPR
  14. Protocol Design
  15. Hyperparameter Optimization
  16. ...and 12 more sections

Key Result

Lemma 1

For the dynamics eq:metroQuant, there exists a class-$\mathcal{K}$ function $\alpha: {\mathbb R}_{\ge 0} \to {\mathbb R}_{\ge 0}$ such that for all $i\in\mathcal{V}$. $\square$

Figures (3)

  • Figure 1: Illustration of $\mathcal{G}$ with $5$ agents. The left and right diagrams depict \ref{['eq:jshare']} and \ref{['eq:mask']}, respectively, for $i=2$ and $j=4$. In this case, $\mathcal{N}_i^+\cap \mathcal{N}_j^+=\{1,2,3,4\}$.
  • Figure 2: Convergence of the hyperparameter estimates $\theta_{i,l}(t)$ and $\theta_{i,s}(t)$ across $20$ agents when using \ref{['eqs:hyper']}. For SARCOS dataset, we only show the estimates corresponding to the first output dimension as a representative case.
  • Figure 3: Monotonic decrease of RMSE as number of iterations $T$ increases and scale factor $\mathsf{L}_z$ decreases.

Theorems & Definitions (11)

  • Example 1
  • Example 2
  • Remark 1
  • Lemma 1: ChamLiuj16
  • Theorem 1
  • Lemma 2
  • Remark 2
  • Definition 1
  • Theorem 2
  • Theorem 3
  • ...and 1 more