Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spin: An Efficient Secure Computation Framework with GPU Acceleration

Wuxuan Jiang, Xiangjun Song, Shenbai Hong, Haijun Zhang, Wenxin Liu, Bo Zhao, Wei Xu, Yi Li

TL;DR

Spin addresses the tension between privacy and efficiency in multi-party computation by delivering a GPU-accelerated framework that supports dishonest-majority security. It introduces novel nonlinear-function algorithms (reciprocal, exponentiation, logarithm) and attention-specific optimizations to accelerate secure CNN training and Transformer inference, backed by RDMA-enabled communication and CPU-GPU hybrid execution. Empirical results show up to $2\times$ speedups in training, improved accuracy relative to state-of-the-art, and substantial efficiency gains in Transformer inference for models with tens of millions of parameters. The work demonstrates practical viability of secure training and inference for large neural networks, with concrete hardware-aware optimizations and a pathway to scaling to larger models and clusters.

Abstract

Accuracy and efficiency remain challenges for multi-party computation (MPC) frameworks. Spin is a GPU-accelerated MPC framework that supports multiple computation parties and a dishonest majority adversarial setup. We propose optimized protocols for non-linear functions that are critical for machine learning, as well as several novel optimizations specific to attention that is the fundamental unit of Transformer models, allowing Spin to perform non-trivial CNNs training and Transformer inference without sacrificing security. At the backend level, Spin leverages GPU, CPU, and RDMA-enabled smart network cards for acceleration. Comprehensive evaluations demonstrate that Spin can be up to $2\times$ faster than the state-of-the-art for deep neural network training. For inference on a Transformer model with 18.9 million parameters, our attention-specific optimizations enable Spin to achieve better efficiency, less communication, and better accuracy.

Spin: An Efficient Secure Computation Framework with GPU Acceleration

TL;DR

Spin addresses the tension between privacy and efficiency in multi-party computation by delivering a GPU-accelerated framework that supports dishonest-majority security. It introduces novel nonlinear-function algorithms (reciprocal, exponentiation, logarithm) and attention-specific optimizations to accelerate secure CNN training and Transformer inference, backed by RDMA-enabled communication and CPU-GPU hybrid execution. Empirical results show up to speedups in training, improved accuracy relative to state-of-the-art, and substantial efficiency gains in Transformer inference for models with tens of millions of parameters. The work demonstrates practical viability of secure training and inference for large neural networks, with concrete hardware-aware optimizations and a pathway to scaling to larger models and clusters.

Abstract

Accuracy and efficiency remain challenges for multi-party computation (MPC) frameworks. Spin is a GPU-accelerated MPC framework that supports multiple computation parties and a dishonest majority adversarial setup. We propose optimized protocols for non-linear functions that are critical for machine learning, as well as several novel optimizations specific to attention that is the fundamental unit of Transformer models, allowing Spin to perform non-trivial CNNs training and Transformer inference without sacrificing security. At the backend level, Spin leverages GPU, CPU, and RDMA-enabled smart network cards for acceleration. Comprehensive evaluations demonstrate that Spin can be up to faster than the state-of-the-art for deep neural network training. For inference on a Transformer model with 18.9 million parameters, our attention-specific optimizations enable Spin to achieve better efficiency, less communication, and better accuracy.
Paper Structure (36 sections, 1 theorem, 3 equations, 6 figures, 10 tables, 5 algorithms)

This paper contains 36 sections, 1 theorem, 3 equations, 6 figures, 10 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Overview
  4. Threat Model
  5. System Design
  6. Building Blocks
  7. Notations
  8. Precomputation Data
  9. Basic Operations
  10. Derived Operations
  11. Non-linear Functions Evaluation
  12. Reciprocal
  13. Exponentiation
  14. Logarithm
  15. Security Analysis
  16. ...and 21 more sections

Key Result

Theorem C.1

In the $F_{\text{ABB}}$-Hybrid model, our reciprocal algorithm alg:ssnr realizes the ideal functionality $F_{\text{Reciprocal}}$ in presence of a semi-honest adversary.

Figures (6)

  • Figure 1: The architecture of Spin. There can exist multiple parties with individual inputs in each task.
  • Figure 2: Pipelined data transfer. The ring labeled 'S' represents the sending data ring buffer, while the ring labeled 'R' represents the receiving data ring buffer.
  • Figure 3: Runtime breakdown.
  • Figure 4: Details of attention blocks.
  • Figure 5: The performance of our integer kernel and pipelined communication solution.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Theorem C.1
  • proof : Proof