Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tempo: Confidentiality Preservation in Cloud-Based Neural Network Training

Rongwu Xu, Zhixuan Fang

TL;DR

Tempo tackles confidentiality in cloud-based neural network training by integrating a master TEE with GPU workers and introducing a permutation-based MM-obfuscation to blind both inputs and weights during offloaded linear algebra. It presents forward and backward passes that minimize encryption overhead through a key-shift mechanism and reuse of encrypted results, supported by verifiable $t$-integrity and an information-theoretic privacy metric. The system is implemented on Intel SGX with Graphene-SGX and evaluated on ResNet-50 and ViT with CIFAR-10, achieving up to roughly 5x speedups over pure-Tee baselines while maintaining accuracy and showing resilience to model theft attacks. By enabling GPU-accelerated, privacy-preserving cloud DL, Tempo offers a practical path toward secure ML as a Service, with extensibility to other TEEs and distributed configurations.

Abstract

Cloud deep learning platforms provide cost-effective deep neural network (DNN) training for customers who lack computation resources. However, cloud systems are often untrustworthy and vulnerable to attackers, leading to growing concerns about model privacy. Recently, researchers have sought to protect data privacy in deep learning by leveraging CPU trusted execution environments (TEEs), which minimize the use of cryptography, but existing works failed to simultaneously utilize the computational resources of GPUs to assist in training and prevent model leakage. This paper presents Tempo, the first cloud-based deep learning system that cooperates with TEE and distributed GPUs for efficient DNN training with model confidentiality preserved. To tackle the challenge of preserving privacy while offloading linear algebraic operations from TEE to GPUs for efficient batch computation, we introduce a customized permutation-based obfuscation algorithm to blind both inputs and model parameters. An optimization mechanism that reduces encryption operations is proposed for faster weight updates during backpropagation to speed up training. We implement Tempo and evaluate it with both training and inference for two prevalent DNNs. Empirical results indicate that Tempo outperforms baselines and offers sufficient privacy protection.

Tempo: Confidentiality Preservation in Cloud-Based Neural Network Training

TL;DR

Tempo tackles confidentiality in cloud-based neural network training by integrating a master TEE with GPU workers and introducing a permutation-based MM-obfuscation to blind both inputs and weights during offloaded linear algebra. It presents forward and backward passes that minimize encryption overhead through a key-shift mechanism and reuse of encrypted results, supported by verifiable -integrity and an information-theoretic privacy metric. The system is implemented on Intel SGX with Graphene-SGX and evaluated on ResNet-50 and ViT with CIFAR-10, achieving up to roughly 5x speedups over pure-Tee baselines while maintaining accuracy and showing resilience to model theft attacks. By enabling GPU-accelerated, privacy-preserving cloud DL, Tempo offers a practical path toward secure ML as a Service, with extensibility to other TEEs and distributed configurations.

Abstract

Cloud deep learning platforms provide cost-effective deep neural network (DNN) training for customers who lack computation resources. However, cloud systems are often untrustworthy and vulnerable to attackers, leading to growing concerns about model privacy. Recently, researchers have sought to protect data privacy in deep learning by leveraging CPU trusted execution environments (TEEs), which minimize the use of cryptography, but existing works failed to simultaneously utilize the computational resources of GPUs to assist in training and prevent model leakage. This paper presents Tempo, the first cloud-based deep learning system that cooperates with TEE and distributed GPUs for efficient DNN training with model confidentiality preserved. To tackle the challenge of preserving privacy while offloading linear algebraic operations from TEE to GPUs for efficient batch computation, we introduce a customized permutation-based obfuscation algorithm to blind both inputs and model parameters. An optimization mechanism that reduces encryption operations is proposed for faster weight updates during backpropagation to speed up training. We implement Tempo and evaluate it with both training and inference for two prevalent DNNs. Empirical results indicate that Tempo outperforms baselines and offers sufficient privacy protection.
Paper Structure (26 sections, 3 theorems, 14 equations, 13 figures, 2 tables, 4 algorithms)

This paper contains 26 sections, 3 theorems, 14 equations, 13 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Deep Neural Network Training
  4. Confidential Computing using TEE and Intel SGX
  5. Problem Statement
  6. Problem Setting
  7. Adversarial Model
  8. Design Goals
  9. $\mathrm{Tempo}$
  10. The MM-obfuscation Algorithm
  11. Privacy-preserving Feedforward
  12. Privacy-preserving Backpropagation
  13. Distributed Training using $\mathrm{Tempo}$
  14. Analysis of $\mathrm{Tempo}$
  15. Proof of Properties of $\mathrm{Tempo}$
  16. ...and 11 more sections

Key Result

Theorem 5.1

The MM-obfuscation algorithm (described in alg:encryption and alg:decryption) is correct, i.e., let $(\mathbf{A}',\mathbf{B}') = \texttt{Enc}(\mathsf{sk},\mathbf{A},\mathbf{B})$ and $\mathbf{C}' = \mathbf{A}'\mathbf{B}'$. Then, $\texttt{Dec}(\mathsf{sk}, \mathbf{C}') = \mathbf{A}\mathbf{B}$.

Figures (13)

  • Figure 1: Overview of the system setup, and communications of $\mathrm{Tempo}$.
  • Figure 2: Workflow of the MM-obfuscation algorithm in $\mathrm{Tempo}$.
  • Figure 3: Backpropagation of DNN training in $\mathrm{Tempo}$.
  • Figure 4: Computations of $\mathrm{Tempo}$ in forward pass at layer $l$
  • Figure 5: Computations of $\mathrm{Tempo}$ in backpropagation at layer $l$
  • ...and 8 more figures

Theorems & Definitions (6)

  • Theorem 5.1: Correctness
  • proof
  • Theorem 5.2: Integrity
  • proof
  • Theorem 5.3: Privacy
  • proof