Dash: Accelerating Distributed Private Convolutional Neural Network Inference with Arithmetic Garbled Circuits

TL;DR

Dash is presented, a fast and distributed private convolutional neural network inference scheme secure against malicious attackers, based purely on arithmetic garbled circuits, and introduces LabelTensors that allow us to leverage the massive parallelity of modern GPUs.

Abstract

The adoption of machine learning solutions is rapidly increasing across all parts of society. As the models grow larger, both training and inference of machine learning models is increasingly outsourced, e.g. to cloud service providers. This means that potentially sensitive data is processed on untrusted platforms, which bears inherent data security and privacy risks. In this work, we investigate how to protect distributed machine learning systems, focusing on deep convolutional neural networks. The most common and best-performing mixed MPC approaches are based on HE, secret sharing, and garbled circuits. They commonly suffer from large performance overheads, big accuracy losses, and communication overheads that grow linearly in the depth of the neural network. To improve on these problems, we present Dash, a fast and distributed private convolutional neural network inference scheme secure against malicious attackers. Building on arithmetic garbling gadgets [BMR16] and fancy-garbling [BCM+19], Dash is based purely on arithmetic garbled circuits. We introduce LabelTensors that allow us to leverage the massive parallelity of modern GPUs. Combined with state-of-the-art garbling optimizations, Dash outperforms previous garbling approaches up to a factor of about 100. Furthermore, we introduce an efficient scaling operation over the residues of the Chinese remainder theorem representation to arithmetic garbled circuits, which allows us to garble larger networks and achieve much higher accuracy than previous approaches. Finally, Dash requires only a single communication round per inference step, regardless of the depth of the neural network, and a very small constant online communication volume.

Figures (9)

• Figure 1: Visualization of our scaling operation in $\mathbb{Z}_{P_3}$.
• Figure 2: System-level architecture of Dash.
• Figure 3: Example workflow of Dash. The first four steps can be pre-computed in an input-independent offline phase. Note that the inference device works on garbled data.
• Figure 4: C++-interface of Dash
• Figure 5: Visualization of the LabelTensor approach in Dash. The length $l$ of a label with modul $p_i$ is defined as $l=\left\lfloor 128 / \log_2(p_i)\right\rfloor$, since a longer label can not be packed into a 128bit chunk which is needed in the permutation used for garbling.