Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacy

Haoqi Wu, Wei Dai, Li Wang, Qiang Yan

TL;DR

Cape tackles the privacy risk of sending prompts to cloud LLMs by introducing a context-aware prompt perturbation mechanism grounded in local differential privacy. It combines a hybrid utility function that leverages both contextual logits and token embedding distances with a bucketized exponential mechanism to efficiently sample replacement tokens over large vocabularies. The approach includes a non-sensitive token set to preserve readability and demonstrates superior privacy-utility trade-offs on text classification and open-ended generation tasks while maintaining practical on-device latency. This work advances privacy-preserving LLM inference with scalable, context-sensitive perturbation that can be deployed without modifying back-end models or requiring secure computation channels.

Abstract

Large Language Models (LLMs) have gained significant popularity due to their remarkable capabilities in text understanding and generation. However, despite their widespread deployment in inference services such as ChatGPT, concerns about the potential leakage of sensitive user data have arisen. Existing solutions primarily rely on privacy-enhancing technologies to mitigate such risks, facing the trade-off among efficiency, privacy, and utility. To narrow this gap, we propose Cape, a context-aware prompt perturbation mechanism based on differential privacy, to enable efficient inference with an improved privacy-utility trade-off. Concretely, we introduce a hybrid utility function that better captures the token similarity. Additionally, we propose a bucketized sampling mechanism to handle large sampling space, which might lead to long-tail phenomenons. Extensive experiments across multiple datasets, along with ablation studies, demonstrate that Cape achieves a better privacy-utility trade-off compared to prior state-of-the-art works.

Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacy

TL;DR

Cape tackles the privacy risk of sending prompts to cloud LLMs by introducing a context-aware prompt perturbation mechanism grounded in local differential privacy. It combines a hybrid utility function that leverages both contextual logits and token embedding distances with a bucketized exponential mechanism to efficiently sample replacement tokens over large vocabularies. The approach includes a non-sensitive token set to preserve readability and demonstrates superior privacy-utility trade-offs on text classification and open-ended generation tasks while maintaining practical on-device latency. This work advances privacy-preserving LLM inference with scalable, context-sensitive perturbation that can be deployed without modifying back-end models or requiring secure computation channels.

Abstract

Large Language Models (LLMs) have gained significant popularity due to their remarkable capabilities in text understanding and generation. However, despite their widespread deployment in inference services such as ChatGPT, concerns about the potential leakage of sensitive user data have arisen. Existing solutions primarily rely on privacy-enhancing technologies to mitigate such risks, facing the trade-off among efficiency, privacy, and utility. To narrow this gap, we propose Cape, a context-aware prompt perturbation mechanism based on differential privacy, to enable efficient inference with an improved privacy-utility trade-off. Concretely, we introduce a hybrid utility function that better captures the token similarity. Additionally, we propose a bucketized sampling mechanism to handle large sampling space, which might lead to long-tail phenomenons. Extensive experiments across multiple datasets, along with ablation studies, demonstrate that Cape achieves a better privacy-utility trade-off compared to prior state-of-the-art works.
Paper Structure (36 sections, 1 theorem, 11 equations, 12 figures, 10 tables, 2 algorithms)

This paper contains 36 sections, 1 theorem, 11 equations, 12 figures, 10 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Transformer-based Inference
  5. Differential Privacy
  6. Private Selection
  7. Threat Model
  8. Design
  9. Context-aware Prompt Perturbation
  10. Utility Function
  11. Sampling Algorithm
  12. Define Non-sensitive Tokens
  13. Experiments
  14. Privacy-Utility Trade-off
  15. Text Classification
  16. ...and 21 more sections

Key Result

Theorem 4.1

The bucketized Exponential Mechanism satisfies $(\epsilon + \epsilon')$-differential privacy, where $\epsilon' = \ln(\underset{i, j}{\mathrm{max}}\frac{|b_i|}{|b_j|})$.

Figures (12)

  • Figure 1: Illustration of existing LLM inference services.
  • Figure 2: Intuition: Drawback of embedding distance alone.
  • Figure 3: High-level workflow of Cape.
  • Figure 4: Cumulative distribution function of sampling probabilities for 'book' in 'This is a good book.' upon GPT2.
  • Figure 5: Privacy-utility trade-offs in terms of privacy attacks vs. accuracy rates by varying $\epsilon \in [1, 20]$. Privacy is measured by privacy scores under empirical attacks.
  • ...and 7 more figures

Theorems & Definitions (4)

  • Definition 3.1: $\epsilon$-Local Differential Privacy ldp-08
  • Definition 3.2: $\epsilon$-Exponential Mechanism (EM) em-07
  • Theorem 4.1
  • proof : Proof of Theorem \ref{['theo:bem']}