Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UFO: Unlocking Ultra-Efficient Quantized Private Inference with Protocol and Algorithm Co-Optimization

Wenxuan Zeng, Chao Yang, Tianshi Xu, Bo Zhang, Changrui Ren, Jin Dong, Meng Li

TL;DR

UFO, a quantized 2PC inference framework that jointly optimizes the 2PC protocols and quantization algorithm and develops a mixed-precision QAT algorithm based on layer sensitivity to optimize model accuracy given communication constraints.

Abstract

Private convolutional neural network (CNN) inference based on secure two-party computation (2PC) suffers from high communication and latency overhead, especially from convolution layers. In this paper, we propose UFO, a quantized 2PC inference framework that jointly optimizes the 2PC protocols and quantization algorithm. UFO features a novel 2PC protocol that systematically combines the efficient Winograd convolution algorithm with quantization to improve inference efficiency. However, we observe that naively combining quantization and Winograd convolution faces the following challenges: 1) From the inference perspective, Winograd transformations introduce extensive additions and require frequent bit width conversions to avoid inference overflow, leading to non-negligible communication overhead; 2) From the training perspective, Winograd transformations introduce weight outliers that make quantization-aware training (QAT) difficult, resulting in inferior model accuracy. To address these challenges, we co-optimize both protocol and algorithm. 1) At the protocol level, we propose a series of graph-level optimizations for 2PC inference to minimize the communication. 2) At the algorithm level, we develop a mixed-precision QAT algorithm based on layer sensitivity to optimize model accuracy given communication constraints. To accommodate the outliers, we further introduce a 2PC-friendly bit re-weighting algorithm to increase the representation range without explicitly increasing bit widths. With extensive experiments, UFO demonstrates 11.7x, 3.6x, and 6.3x communication reduction with 1.29%, 1.16%, and 1.29% higher accuracy compared to state-of-the-art frameworks SiRNN, COINN, and CoPriv, respectively.

UFO: Unlocking Ultra-Efficient Quantized Private Inference with Protocol and Algorithm Co-Optimization

TL;DR

UFO, a quantized 2PC inference framework that jointly optimizes the 2PC protocols and quantization algorithm and develops a mixed-precision QAT algorithm based on layer sensitivity to optimize model accuracy given communication constraints.

Abstract

Private convolutional neural network (CNN) inference based on secure two-party computation (2PC) suffers from high communication and latency overhead, especially from convolution layers. In this paper, we propose UFO, a quantized 2PC inference framework that jointly optimizes the 2PC protocols and quantization algorithm. UFO features a novel 2PC protocol that systematically combines the efficient Winograd convolution algorithm with quantization to improve inference efficiency. However, we observe that naively combining quantization and Winograd convolution faces the following challenges: 1) From the inference perspective, Winograd transformations introduce extensive additions and require frequent bit width conversions to avoid inference overflow, leading to non-negligible communication overhead; 2) From the training perspective, Winograd transformations introduce weight outliers that make quantization-aware training (QAT) difficult, resulting in inferior model accuracy. To address these challenges, we co-optimize both protocol and algorithm. 1) At the protocol level, we propose a series of graph-level optimizations for 2PC inference to minimize the communication. 2) At the algorithm level, we develop a mixed-precision QAT algorithm based on layer sensitivity to optimize model accuracy given communication constraints. To accommodate the outliers, we further introduce a 2PC-friendly bit re-weighting algorithm to increase the representation range without explicitly increasing bit widths. With extensive experiments, UFO demonstrates 11.7x, 3.6x, and 6.3x communication reduction with 1.29%, 1.16%, and 1.29% higher accuracy compared to state-of-the-art frameworks SiRNN, COINN, and CoPriv, respectively.
Paper Structure (52 sections, 4 theorems, 12 equations, 18 figures, 6 tables, 1 algorithm)

This paper contains 52 sections, 4 theorems, 12 equations, 18 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries and Backgrounds
  3. Threat Model
  4. Private Inference
  5. Secret Sharing and OT-based Linear Layer
  6. Model Quantization
  7. Winograd Convolution
  8. Motivations
  9. Motivation 1: the total communication of OT-based 2PC inference is determined by both the bit widths and the number of multiplications in linear layers
  10. Motivation 2: a naive combination of Winograd convolution and quantization provides limited communication reduction for 2PC inference (induced by feature and output transformation)
  11. Motivation 3: Winograd transformations introduce weight outliers and make low-precision QAT difficult (induced by weight transformation)
  12. Paper Organization
  13. Inference Protocol Optimization
  14. New Protocol: Quantized Winograd Convolution
  15. Protocol Framework
  16. ...and 37 more sections

Key Result

Lemma 1

Consider $Y = XB$ in feature transformation, where $B$ is the transformation matrix. For each element $Y_{i, j}$, its magnitude can always be bounded by where $||\cdot||_1$ is the $\ell_1$-norm, $\lceil\cdot\rceil$ is the ceiling operation.

Figures (18)

  • Figure 1: Motivating inspirations. (a) Total communication and (b) online communication breakdown on the ResNet-50 block profiled with CrypTFlow2 (CTF2) rathee2020cryptflow2 (uniform 37-bit) and SiRNN rathee2021sirnn (low precision); (c) weight distributions in regular and Winograd convolution; (d) z-score (defined as the ratio between max-average and standard deviation) indicates weight outliers consistently exist after Winograd transformation across different layers.
  • Figure 2: Comparison between OT and HE with quantization. HE requires much higher ciphertext (CT) bit widths and cannot utilize low precision, e.g., 4 bits.
  • Figure 3: Overall pipeline of Winograd convolution with low-precision quantization.
  • Figure 4: Protocol of OT-based linear layer, including a pre-processing stage and an online stage to process client's input.
  • Figure 5: An illustration of GEMM $Y=WX$.
  • ...and 13 more figures

Theorems & Definitions (4)

  • Lemma 1
  • Proposition 1
  • Proposition 2
  • Proposition 3