A generic framework for privacy preserving deep learning

TL;DR

This work introduces a general, PyTorch-based framework for privacy-preserving deep learning that unifies Federated Learning, Secure Multiparty Computation, and Differential Privacy through a tensor-chain abstraction. It details a standardized tensor operation protocol, an MPCTensor built atop the SPDZ protocol, and a DP-SGD implementation with a privacy accountant, all designed to be extensible via plug-in privacy methods. Experimental results demonstrate function, but reveal substantial training-time overhead and a clear privacy-utility trade-off, highlighting the need for efficiency and robust security measures in MPC. Overall, the framework represents a significant step toward accessible, general-purpose privacy-preserving DL capable of integrating diverse privacy techniques within a common API.

Abstract

We detail a new framework for privacy preserving deep learning and discuss its assets. The framework puts a premium on ownership and secure processing of data and introduces a valuable representation based on chains of commands and tensors. This abstraction allows one to implement complex privacy preserving constructs such as Federated Learning, Secure Multiparty Computation, and Differential Privacy while still exposing a familiar deep learning API to the end-user. We report early results on the Boston Housing and Pima Indian Diabetes datasets. While the privacy features apart from Differential Privacy do not impact the prediction accuracy, the current implementation of the framework introduces a significant overhead in performance, which will be addressed at a later stage of the development. We believe this work is an important milestone introducing the first reliable, general framework for privacy preserving deep learning.

Figures (3)

• Figure 1: General structure of a tensor chain
• Figure 2: Impact of sending a tensor on the local and remote chains
• Figure 3: Chain structure of a SPDZ tensor