Private Transformer Inference in MLaaS: A Survey

TL;DR

The work addresses the privacy challenges of Transformer inference in MLaaS by surveying Private Transformer Inference (PTI) methods that rely on secure multi-party computation (MPC) and homomorphic encryption (HE). It introduces a structured taxonomy of Transformer layers (linear vs non-linear) and an evaluation framework to analyze trade-offs in communication, runtime, and accuracy. The survey synthesizes state-of-the-art PTI solutions, detailing approaches for linear layers (MPC and HE) and non-linear layers (Softmax, GELU, LayerNorm), and reviews experimental results across models and tasks. The findings highlight substantial accuracy and efficiency challenges, especially for non-linear components, and point to future directions such as GPU-accelerated cryptography and broader generation-task support for practical, privacy-preserving AI services.

Abstract

Transformer models have revolutionized AI, powering applications like content generation and sentiment analysis. However, their deployment in Machine Learning as a Service (MLaaS) raises significant privacy concerns, primarily due to the centralized processing of sensitive user data. Private Transformer Inference (PTI) offers a solution by utilizing cryptographic techniques such as secure multi-party computation and homomorphic encryption, enabling inference while preserving both user data and model privacy. This paper reviews recent PTI advancements, highlighting state-of-the-art solutions and challenges. We also introduce a structured taxonomy and evaluation framework for PTI, focusing on balancing resource efficiency with privacy and bridging the gap between high-performance inference and data privacy.

Figures (2)

• Figure 1: MLaaS without privacy protection
• Figure 2: The architecture of a transformer encoder