Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bicoptor 2.0: Addressing Challenges in Probabilistic Truncation for Enhanced Privacy-Preserving Machine Learning

Lijing Zhou, Qingrui Song, Su Zhang, Ziyu Wang, Xianggui Wang, Yong Li

TL;DR

This paper introduces the first non-interactive deterministic truncation protocol, replacing the original probabilistic truncation Protocol, and provides a guideline to reduce computational and communication overhead by using only a portion of the bits of the input, i.e., the key bits, for DReLU operations based on different model parameters.

Abstract

This paper primarily focuses on analyzing the problems and proposing solutions for the probabilistic truncation protocol in existing PPML works from the perspectives of accuracy and efficiency. In terms of accuracy, we reveal that precision selections recommended in some of the existing works are incorrect. We conduct a thorough analysis of their open-source code and find that their errors were mainly due to simplified implementation, more specifically, fixed numbers are used instead of random numbers in probabilistic truncation protocols. Based on this, we provide a detailed theoretical analysis to validate our views. We propose a solution and a precision selection guideline for future works. Regarding efficiency, we identify limitations in the state-of-the-art comparison protocol, Bicoptor's (S\&P 2023) DReLU protocol, which relies on the probabilistic truncation protocol and is heavily constrained by the security parameter to avoid errors, significantly impacting the protocol's performance. To address these challenges, we introduce the first non-interactive deterministic truncation protocol, replacing the original probabilistic truncation protocol. Additionally, we design a non-interactive modulo switch protocol to enhance the protocol's security. Finally, we provide a guideline to reduce computational and communication overhead by using only a portion of the bits of the input, i.e., the key bits, for DReLU operations based on different model parameters. With the help of key bits, the performance of our DReLU protocol is further improved. We evaluate the performance of our protocols on three GPU servers, and achieve a 10x improvement in DReLU protocol, and a 6x improvement in the ReLU protocol over the state-of-the-art work Piranha-Falcon (USENIX Sec 22). Overall, the performance of our end-to-end (E2E) privacy-preserving machine learning (PPML) inference is improved by 3-4 times.

Bicoptor 2.0: Addressing Challenges in Probabilistic Truncation for Enhanced Privacy-Preserving Machine Learning

TL;DR

This paper introduces the first non-interactive deterministic truncation protocol, replacing the original probabilistic truncation Protocol, and provides a guideline to reduce computational and communication overhead by using only a portion of the bits of the input, i.e., the key bits, for DReLU operations based on different model parameters.

Abstract

This paper primarily focuses on analyzing the problems and proposing solutions for the probabilistic truncation protocol in existing PPML works from the perspectives of accuracy and efficiency. In terms of accuracy, we reveal that precision selections recommended in some of the existing works are incorrect. We conduct a thorough analysis of their open-source code and find that their errors were mainly due to simplified implementation, more specifically, fixed numbers are used instead of random numbers in probabilistic truncation protocols. Based on this, we provide a detailed theoretical analysis to validate our views. We propose a solution and a precision selection guideline for future works. Regarding efficiency, we identify limitations in the state-of-the-art comparison protocol, Bicoptor's (S\&P 2023) DReLU protocol, which relies on the probabilistic truncation protocol and is heavily constrained by the security parameter to avoid errors, significantly impacting the protocol's performance. To address these challenges, we introduce the first non-interactive deterministic truncation protocol, replacing the original probabilistic truncation protocol. Additionally, we design a non-interactive modulo switch protocol to enhance the protocol's security. Finally, we provide a guideline to reduce computational and communication overhead by using only a portion of the bits of the input, i.e., the key bits, for DReLU operations based on different model parameters. With the help of key bits, the performance of our DReLU protocol is further improved. We evaluate the performance of our protocols on three GPU servers, and achieve a 10x improvement in DReLU protocol, and a 6x improvement in the ReLU protocol over the state-of-the-art work Piranha-Falcon (USENIX Sec 22). Overall, the performance of our end-to-end (E2E) privacy-preserving machine learning (PPML) inference is improved by 3-4 times.
Paper Structure (28 sections, 15 theorems, 26 equations, 5 figures, 9 tables, 9 algorithms)

This paper contains 28 sections, 15 theorems, 26 equations, 5 figures, 9 tables, 9 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Our Contributions
  4. Preliminaries
  5. System Settings
  6. Notations
  7. Cut Function and Truncation Protocol
  8. The Truncation Protocols Proposed in Prior Work Used in This Paper
  9. In-depth Analysis of $e_1$ in Probabilistic Truncation Protocols
  10. Impact of $e_1$ on Accuracy in PPML and Propoesed Solutions
  11. The Implementation Bug Related to $e_1$ in Existing Works
  12. Proposed Solution to Address the Implementation Bug Related to $e_1$
  13. Impact of $e_1$ on Efficiency in PPML and Propoesed Solutions
  14. A Non-interactive Deterministic Truncation Protocol
  15. The Principle of Sign Determination
  16. ...and 13 more sections

Key Result

Theorem 1

In a ring $\mathbb{Z}_{2^\ell}$, let $x\in[0,2^{\ell_x})\bigcup(2^\ell-2^{\ell_x},2^\ell)$, where $\ell>\ell_x + 1$. Then the outputs of Alg. alg:smltrc satisfy the following results with probability $1-\frac{1}{2^{\ell-\ell_x-1}}$, where $\mathsf{bit}:=\{0,1\}$.

Figures (5)

  • Figure 1: The Overview of This Paper. "Prob. Trc." stands for Probabilistic Truncation; "Determ. Trc." stands for Deterministic Truncation; "Trc-then-Mult" stands for truncate then multiply.
  • Figure 2: Comparison of the effect of different precisions with fixed/random numbers on inference accuracy for CIFAR10_AlexNet, Tiny_AlexNet, CIFAR10_VGG16, and Tiny_VGG16. The experiments are carried using ring $\mathbb{Z}_{2^{64}}$ or $\mathbb{Z}_{2^{32}}$ in $\text{ABY}^3$ truncation protocol in P-Falcon WWP22piranha-code. Entries with (f) indicate the use of fixed numbers, while entries with (r) indicate the use of random numbers.
  • Figure 3: The Overview of Bicoptor 2.0 DReLU Protocol.
  • Figure 4: Performance comparisons of P-Falcon WWP22piranha-code and Bicoptor 2.0 DReLU and ReLU protocols on the different networks and batch sizes. (Graph)
  • Figure 5: Runtime comparison of P-Falcon WWP22piranha-code and Bicoptor 2.0 ReLU protocols and the inference for various models in LAN1. In Bicoptor 2.0, $\ell_x=7$ for DReLU, and both UBL and RSS modes are evaluated. The batch size is selected to ensure each participant spends around 8 GiB GPU memory.

Theorems & Definitions (16)

  • Theorem 1
  • Definition 1
  • Theorem 2
  • Corollary 1
  • Theorem 3
  • Theorem 4
  • Lemma 1
  • Corollary 2
  • Lemma 2
  • Proposition 1
  • ...and 6 more