Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OPAF: Optimized Secure Two-Party Computation Protocols for Nonlinear Activation Functions in Recurrent Neural Network

Qian Feng, Zhihua Xia, Zhifeng Xu, Jiasi Weng, Jian Weng

TL;DR

This paper tackles privacy-preserving RNN inference by advancing secure two-party computation for nonlinear activations, specifically exponential, sigmoid, and tanh. It introduces a divide-and-conquer exponential protocol with most work performed locally, and symmetry-based optimizations plus a small input-tuning trick to streamline sigmoid and tanh, including a negative-input exponential variant. The proposed protocols demonstrate state-of-the-art precision (low ULP errors) and substantial efficiency gains in communication and run-time, validated on end-to-end RNN inference (FastGRNN on Google-30) and across LAN/WAN settings. These results enhance the practicality of privacy-preserving DNNs and offer generalizable techniques for efficient secure evaluation of symmetric nonlinear functions.

Abstract

Deep neural network (DNN) typically involves convolutions, pooling, and activation function. Due to the growing concern about privacy, privacy-preserving DNN becomes a hot research topic. Generally, the convolution and pooling operations can be supported by additive homomorphic and secure comparison, but the secure implementation of activation functions is not so straightforward for the requirements of accuracy and efficiency, especially for the non-linear ones such as exponential, sigmoid, and tanh functions. This paper pays a special attention to the implementation of such non-linear functions in semi-honest model with two-party settings, for which SIRNN is the current state-of-the-art. Different from previous works, we proposed improved implementations for these functions by using their intrinsic features as well as worthy tiny tricks. At first, we propose a novel and efficient protocol for exponential function by using a divide-and-conquer strategy with most of the computations executed locally. Exponential protocol is widely used in machine learning tasks such as Poisson regression, and is also a key component of sigmoid and tanh functions. Next, we take advantage of the symmetry of sigmoid and Tanh, and fine-tune the inputs to reduce the 2PC building blocks, which helps to save overhead and improve performance. As a result, we implement these functions with fewer fundamental building blocks. The comprehensive evaluations show that our protocols achieve state-of-the-art precision while reducing run-time by approximately 57%, 44%, and 42% for exponential (with only negative inputs), sigmoid, and Tanh functions, respectively.

OPAF: Optimized Secure Two-Party Computation Protocols for Nonlinear Activation Functions in Recurrent Neural Network

TL;DR

This paper tackles privacy-preserving RNN inference by advancing secure two-party computation for nonlinear activations, specifically exponential, sigmoid, and tanh. It introduces a divide-and-conquer exponential protocol with most work performed locally, and symmetry-based optimizations plus a small input-tuning trick to streamline sigmoid and tanh, including a negative-input exponential variant. The proposed protocols demonstrate state-of-the-art precision (low ULP errors) and substantial efficiency gains in communication and run-time, validated on end-to-end RNN inference (FastGRNN on Google-30) and across LAN/WAN settings. These results enhance the practicality of privacy-preserving DNNs and offer generalizable techniques for efficient secure evaluation of symmetric nonlinear functions.

Abstract

Deep neural network (DNN) typically involves convolutions, pooling, and activation function. Due to the growing concern about privacy, privacy-preserving DNN becomes a hot research topic. Generally, the convolution and pooling operations can be supported by additive homomorphic and secure comparison, but the secure implementation of activation functions is not so straightforward for the requirements of accuracy and efficiency, especially for the non-linear ones such as exponential, sigmoid, and tanh functions. This paper pays a special attention to the implementation of such non-linear functions in semi-honest model with two-party settings, for which SIRNN is the current state-of-the-art. Different from previous works, we proposed improved implementations for these functions by using their intrinsic features as well as worthy tiny tricks. At first, we propose a novel and efficient protocol for exponential function by using a divide-and-conquer strategy with most of the computations executed locally. Exponential protocol is widely used in machine learning tasks such as Poisson regression, and is also a key component of sigmoid and tanh functions. Next, we take advantage of the symmetry of sigmoid and Tanh, and fine-tune the inputs to reduce the 2PC building blocks, which helps to save overhead and improve performance. As a result, we implement these functions with fewer fundamental building blocks. The comprehensive evaluations show that our protocols achieve state-of-the-art precision while reducing run-time by approximately 57%, 44%, and 42% for exponential (with only negative inputs), sigmoid, and Tanh functions, respectively.
Paper Structure (22 sections, 26 equations, 3 figures, 11 tables, 4 algorithms)

This paper contains 22 sections, 26 equations, 3 figures, 11 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Fixed-Point Representation
  5. ULP Errors
  6. Secret Sharing
  7. 2PC and Threat Model
  8. 2PC Functionalities
  9. The Proposed 2PC Protocols
  10. Exponential Function
  11. Exponential Function with Negative Input only
  12. Sigmoid Function
  13. Tanh Function
  14. Implementation
  15. Evaluation
  16. ...and 7 more sections

Figures (3)

  • Figure 1: The communication traffic of $\mathcal{F}_\textsf{expn}$ with different scale $s$.
  • Figure 2: The communication traffic of $\mathcal{F}_\textsf{sigmoid}$ with different scale $s$.
  • Figure 3: The communication traffic of $\mathcal{F}_\textsf{Tanh}$ with different scale $s$.