Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PermLLM: Private Inference of Large Language Models within 3 Seconds under WAN

Fei Zheng, Chaochao Chen, Zhongxuan Han, Xiaolin Zheng

TL;DR

PermLLM tackles private inference for large language models by combining random permutation-based nonlinear evaluation with optimized secret-sharing and BFV-based retrieval. It operates in a two-party setting with an offline helper for precomputation, outsourcing nonlinear computations to the user in a permuted form to preserve privacy. Implemented on ChatGLM-6B, PermLLM achieves roughly 3 seconds per token over WAN (10 ms RTT, 1 Gbps) with roughly 20 MB of traffic, markedly outperforming prior MPC-based solutions. The approach also introduces practical optimizations, such as public parameter exposure and real-number secret sharing, while providing a security analysis that emphasizes practical privacy guarantees despite not achieving information-theoretic security for random permutation. Overall, the work demonstrates that fast, private LLM inference is feasible for real-world deployment under realistic network conditions.

Abstract

The emergence of ChatGPT marks the arrival of the large language model (LLM) era. While LLMs demonstrate their power in a variety of fields, they also raise serious privacy concerns as the users' queries are sent to the model provider. On the other side, deploying the LLM on the user's device will also leak all the model data. Existing methods based on secure multiparty computation (MPC) managed to protect both the privacy of the model parameters and user queries. However, they require gigabytes of data transfer and several minutes to generate just one token, making them impractical for most real-world applications. To improve the efficiency of private LLM inference, we propose PermLLM, which accelerates the evaluation of non-linear functions using secure random permutation. Along with the optimized secret sharing protocols and homomorphic encryption, PermLLM achieves two-party private inference of the ChatGLM-6B model at the speed of around 3s/token, under a realistic network setting (10ms RTT and 1Gbps bandwidth), which is magnitudes faster than existing MPC solutions.

PermLLM: Private Inference of Large Language Models within 3 Seconds under WAN

TL;DR

PermLLM tackles private inference for large language models by combining random permutation-based nonlinear evaluation with optimized secret-sharing and BFV-based retrieval. It operates in a two-party setting with an offline helper for precomputation, outsourcing nonlinear computations to the user in a permuted form to preserve privacy. Implemented on ChatGLM-6B, PermLLM achieves roughly 3 seconds per token over WAN (10 ms RTT, 1 Gbps) with roughly 20 MB of traffic, markedly outperforming prior MPC-based solutions. The approach also introduces practical optimizations, such as public parameter exposure and real-number secret sharing, while providing a security analysis that emphasizes practical privacy guarantees despite not achieving information-theoretic security for random permutation. Overall, the work demonstrates that fast, private LLM inference is feasible for real-world deployment under realistic network conditions.

Abstract

The emergence of ChatGPT marks the arrival of the large language model (LLM) era. While LLMs demonstrate their power in a variety of fields, they also raise serious privacy concerns as the users' queries are sent to the model provider. On the other side, deploying the LLM on the user's device will also leak all the model data. Existing methods based on secure multiparty computation (MPC) managed to protect both the privacy of the model parameters and user queries. However, they require gigabytes of data transfer and several minutes to generate just one token, making them impractical for most real-world applications. To improve the efficiency of private LLM inference, we propose PermLLM, which accelerates the evaluation of non-linear functions using secure random permutation. Along with the optimized secret sharing protocols and homomorphic encryption, PermLLM achieves two-party private inference of the ChatGLM-6B model at the speed of around 3s/token, under a realistic network setting (10ms RTT and 1Gbps bandwidth), which is magnitudes faster than existing MPC solutions.
Paper Structure (20 sections, 1 theorem, 1 equation, 3 figures, 2 tables, 5 algorithms)

This paper contains 20 sections, 1 theorem, 1 equation, 3 figures, 2 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Structure of LLM
  5. Additive Secret Sharing
  6. Secure Building Blocks
  7. Security Setting
  8. Linear Computations
  9. Nonlinear Computation
  10. Next Token Retrieval
  11. Optimizing PermLLM
  12. Making Some Parameters Public
  13. Secret Sharing on Real Numbers
  14. Security Analysis
  15. Random Permutation
  16. ...and 5 more sections

Key Result

Theorem 6.1

If $\mathbb E[(\langle x\rangle_i - x)^2] \ge K^2 \mathbb E[x^2]$, and $\mathbb E[(\langle y \rangle_i - y)^2] \ge K^2 \mathbb E[y^2]$ for $i = 0, 1$, and $\langle z\rangle = \mathsf{SecureMul}_F(x, y)$ or $\mathsf{SecureMul}_G(x, y)$, where the beaver triples is generated in the way described in se

Figures (3)

  • Figure 1: Transformer layer structure.
  • Figure 2: Time consumption and communication size of the ChatGLM-6B private inference.
  • Figure : SecurePerm

Theorems & Definitions (2)

  • Theorem 6.1: Noise scale after multiplication
  • proof