Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Secure Transformer Inference Protocol

Mu Yuan, Lan Zhang, Xiang-Yang Li

TL;DR

STIP addresses the security-efficiency gap in Transformer inference by introducing a three-party threat model that separates model development from hosting. It relies on a feature-space permutation and a semi-symmetric protection scheme to achieve lossless inference while keeping model parameters and on-device data private, with theoretical guarantees and empirical validation. Experiments across language, multi-modal, and MoE variants show STIP attains throughput comparable to unprotected full-cloud inference and dramatically outperforms secure two-party protocols by large margins, while quantifying privacy leakage via distance-correlation metrics. The approach is designed to fit production workflows and frameworks, though its three-party assumptions may limit applicability in some cloud-provider scenarios and does not directly address training privacy.

Abstract

Security of model parameters and user data is critical for Transformer-based services, such as ChatGPT. While recent strides in secure two-party protocols have successfully addressed security concerns in serving Transformer models, their adoption is practically infeasible due to the prohibitive cryptographic overheads involved. Drawing insights from our hands-on experience in developing two real-world Transformer-based services, we identify the inherent efficiency bottleneck in the two-party assumption. To overcome this limitation, we propose a novel three-party threat model. Within this framework, we design a semi-symmetric permutation-based protection scheme and present STIP, the first secure Transformer inference protocol without any inference accuracy loss. Experiments on representative Transformer models in real systems show that STIP has practical security and outperforms state-of-the-art secure two-party protocols in efficiency by millions of times.

Secure Transformer Inference Protocol

TL;DR

STIP addresses the security-efficiency gap in Transformer inference by introducing a three-party threat model that separates model development from hosting. It relies on a feature-space permutation and a semi-symmetric protection scheme to achieve lossless inference while keeping model parameters and on-device data private, with theoretical guarantees and empirical validation. Experiments across language, multi-modal, and MoE variants show STIP attains throughput comparable to unprotected full-cloud inference and dramatically outperforms secure two-party protocols by large margins, while quantifying privacy leakage via distance-correlation metrics. The approach is designed to fit production workflows and frameworks, though its three-party assumptions may limit applicability in some cloud-provider scenarios and does not directly address training privacy.

Abstract

Security of model parameters and user data is critical for Transformer-based services, such as ChatGPT. While recent strides in secure two-party protocols have successfully addressed security concerns in serving Transformer models, their adoption is practically infeasible due to the prohibitive cryptographic overheads involved. Drawing insights from our hands-on experience in developing two real-world Transformer-based services, we identify the inherent efficiency bottleneck in the two-party assumption. To overcome this limitation, we propose a novel three-party threat model. Within this framework, we design a semi-symmetric permutation-based protection scheme and present STIP, the first secure Transformer inference protocol without any inference accuracy loss. Experiments on representative Transformer models in real systems show that STIP has practical security and outperforms state-of-the-art secure two-party protocols in efficiency by millions of times.
Paper Structure (31 sections, 1 theorem, 11 equations, 9 figures, 5 tables, 1 algorithm)

This paper contains 31 sections, 1 theorem, 11 equations, 9 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Device-Cloud Collaboration
  4. Secure Two-Party Inference
  5. Align with Real-World Applications: Three-Party Threat Model
  6. Design Space
  7. Challenges
  8. Prohibitive Cryptographic Overhead
  9. Attack Surface Vulnerability
  10. Definition
  11. Transformer Inference
  12. Three-Party Setting and Threat Model
  13. Design
  14. Feature Space Permutation
  15. Protocol
  16. ...and 16 more sections

Key Result

Theorem 1

$F_{\theta '}(x\pi)\pi_c^T = F_{\theta}(x)$.

Figures (9)

  • Figure 1: Three-Party Threat Model
  • Figure 2: Latency and communication overheads of GPT2-124m model.
  • Figure 3: Inference workflow of original Transformer.
  • Figure 4: Illustration of feature-space parameter transformation. Colors represent the use of different permutation matrices.
  • Figure 5: Overview of STIP.
  • ...and 4 more figures

Theorems & Definitions (1)

  • Theorem 1