Split-and-Denoise: Protect large language model inference with local differential privacy

TL;DR

Split-N-Denoise (SnD) tackles privacy in LLM inference for Embedding-as-a-Service by splitting the model to run the token embedding layer on the client and applying a client-side denoising step after server processing. It employs $d_ Chi$-privacy with Laplacian noise and norm clipping to privatize token representations, and a pre-trained denoising network distributed to users to recover high-utility embeddings without exposing private data. The denoise model is trained on public data to avoid data leakage, enabling privacy-preserving inference with minimal changes to the underlying LLM. Across BERT, GPT-2, and T5, SnD consistently improves the privacy-utility tradeoff compared with baselines, maintaining competitive downstream performance while achieving strong privacy guarantees and substantial computational/communication efficiency relative to cryptographic approaches.

Abstract

Large Language Models (LLMs) excel in natural language understanding by capturing hidden semantics in vector space. This process enriches the value of text embeddings for various downstream tasks, thereby fostering the Embedding-as-a-Service (EaaS) business model. However, the risk of privacy leakage due to direct text transmission to servers remains a critical concern. To address this, we introduce Split-N-Denoise (SnD), an private inference framework that splits the model to execute the token embedding layer on the client side at minimal computational cost. This allows the client to introduce noise prior to transmitting the embeddings to the server, and subsequently receive and denoise the perturbed output embeddings for downstream tasks. Our approach is designed for the inference stage of LLMs and requires no modifications to the model parameters. Extensive experiments demonstrate SnD's effectiveness in optimizing the privacy-utility tradeoff across various LLM architectures and diverse downstream tasks. The results reveal an improvement in performance under the same privacy budget compared to the baselines by over 10\% on average, offering clients a privacy-preserving solution for local privacy protection.

Figures (22)

• Figure 1: Overview of our privacy-preserving SnD framework. Users first obtain an initial embedding from a local encoder, followed by a noise addition via the privatization module. This privatized embedding is then transmitted to the server for processing. Upon completion, users receive a noised output, which is subsequently refined using a pre-trained denoising model to achieve an optimal balance between privacy and utility.
• Figure 2: Estimated mutual information (MI) and embedding inversion attack accuracy under varying $\eta$. MI and attack accuracies approach $0$ under $\eta\leq 0.1$, $\eta\leq 50$, and $\eta\leq 10$ for T5, BERT, and GPT2 models, respectively.
• Figure 3: Architecture of denoise model.The denoise model accepts the noised output embedding from the LLM model, in conjunction with the raw token embedding and noise matrix, as input. Through multiple transformers, the model learns to denoise, ultimately producing a denoised output embedding to augment the performance of downstream tasks.
• Figure 4: Bert
• Figure 5: GPT

Theorems & Definitions (10)

• Definition 3.1: ($\epsilon, \delta$)-Differential Privacy
• Definition 3.2: $d_\chi$-privacy
• Theorem 3.3
• Proposition 3.4
• Remark 3.5
• Lemma 1.1
• proof
• proof
• Lemma 1.2
• proof