Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

InferDPT: Privacy-Preserving Inference for Black-box Large Language Model

Meng Tong, Kejiang Chen, Jie Zhang, Yuang Qi, Weiming Zhang, Nenghai Yu, Tianwei Zhang, Zhikun Zhang

TL;DR

InferDPT tackles privacy risks in prompt-based, black-box LLM inference by introducing a two-module framework: a perturbation module that applies local differential privacy, and an extraction module that distills and aligns perturbed generation with the raw document. The core novelty is RANTEXT, a random-adjacency local differential privacy mechanism that uses random adjacent embeddings and the exponential mechanism to perturb tokens, delivering superior privacy-utility trade-offs against embedding inversion and GPT-based attacks. Across three open-ended generation datasets and multiple attack scenarios, InferDPT achieves generation quality close to non-private GPT-4, while RANTEXT provides strong privacy protection (e.g., >90% against embedding revision at ε = 6.0) and outperforms SANTEXT+ and CUSTEXT+. The work demonstrates practical viability for privacy-preserving inference in black-box LLMs and outlines future directions to close MAUVE gaps and optimize the local extraction component for broader deployment.

Abstract

Large language models (LLMs), like ChatGPT, have greatly simplified text generation tasks. However, they have also raised concerns about privacy risks such as data leakage and unauthorized data collection. Existing solutions for privacy-preserving inference face practical challenges related to computation time and communication costs. In this paper, we propose InferDPT, the first practical framework for the privacy-preserving Inference of black-box LLMs, implementing Differential Privacy in Text generation. InferDPT comprises two key modules: the "perturbation module" utilizes the exponential mechanism to generate a perturbed prompt, facilitating privacy-preserving inference with black-box LLMs, and the "extraction module", inspired by knowledge distillation and retrieval-augmented generation, extracts coherent and consistent text from the perturbed generation result, ensuring successful text generation completion. To address privacy concerns related to previous exponential mechanisms' susceptibility to embedding revision attacks, we introduce RANTEXT, a novel differential privacy mechanism integrated into the perturbation module of InferDPT, which introduces the concept of "RANdom adjacency" for TEXT perturbation within the prompt. Experimental results across three datasets demonstrate that the text generation quality of InferDPT is comparable to that of non-private GPT-4, and RANTEXT surpasses existing state-of-the-art mechanisms, namely, SANTEXT+ and CUSTEXT+ in the trade-off between privacy and utility. Even with an privacy parameter epsilon value of 6.0, RANTEXT achieves an average privacy protection rate exceeding 90% against embedding revision attacks, which is 0.58 times higher than that of SANTEXT+ and 3.35 times higher than that of CUSTEXT+.

InferDPT: Privacy-Preserving Inference for Black-box Large Language Model

TL;DR

InferDPT tackles privacy risks in prompt-based, black-box LLM inference by introducing a two-module framework: a perturbation module that applies local differential privacy, and an extraction module that distills and aligns perturbed generation with the raw document. The core novelty is RANTEXT, a random-adjacency local differential privacy mechanism that uses random adjacent embeddings and the exponential mechanism to perturb tokens, delivering superior privacy-utility trade-offs against embedding inversion and GPT-based attacks. Across three open-ended generation datasets and multiple attack scenarios, InferDPT achieves generation quality close to non-private GPT-4, while RANTEXT provides strong privacy protection (e.g., >90% against embedding revision at ε = 6.0) and outperforms SANTEXT+ and CUSTEXT+. The work demonstrates practical viability for privacy-preserving inference in black-box LLMs and outlines future directions to close MAUVE gaps and optimize the local extraction component for broader deployment.

Abstract

Large language models (LLMs), like ChatGPT, have greatly simplified text generation tasks. However, they have also raised concerns about privacy risks such as data leakage and unauthorized data collection. Existing solutions for privacy-preserving inference face practical challenges related to computation time and communication costs. In this paper, we propose InferDPT, the first practical framework for the privacy-preserving Inference of black-box LLMs, implementing Differential Privacy in Text generation. InferDPT comprises two key modules: the "perturbation module" utilizes the exponential mechanism to generate a perturbed prompt, facilitating privacy-preserving inference with black-box LLMs, and the "extraction module", inspired by knowledge distillation and retrieval-augmented generation, extracts coherent and consistent text from the perturbed generation result, ensuring successful text generation completion. To address privacy concerns related to previous exponential mechanisms' susceptibility to embedding revision attacks, we introduce RANTEXT, a novel differential privacy mechanism integrated into the perturbation module of InferDPT, which introduces the concept of "RANdom adjacency" for TEXT perturbation within the prompt. Experimental results across three datasets demonstrate that the text generation quality of InferDPT is comparable to that of non-private GPT-4, and RANTEXT surpasses existing state-of-the-art mechanisms, namely, SANTEXT+ and CUSTEXT+ in the trade-off between privacy and utility. Even with an privacy parameter epsilon value of 6.0, RANTEXT achieves an average privacy protection rate exceeding 90% against embedding revision attacks, which is 0.58 times higher than that of SANTEXT+ and 3.35 times higher than that of CUSTEXT+.
Paper Structure (31 sections, 35 equations, 10 figures, 15 tables, 2 algorithms)

This paper contains 31 sections, 35 equations, 10 figures, 15 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Large Language Models
  4. (Local) Differential Privacy& Exponential Mechanism
  5. Problem Statement
  6. Threat Model
  7. Existing Solutions and Limitations
  8. The InferDPT Framework
  9. Overview
  10. Key Observations
  11. Perturbation Module
  12. Extraction Module
  13. The RANTEXT Mechanism
  14. Overview
  15. Compute Random Adjacency Lists
  16. ...and 16 more sections

Figures (10)

  • Figure 1: The illustration of potential privacy leakage and a solution via InferDPT when a user employs black-box LLMs for text generation tasks.
  • Figure 2: The overview of InferDPT. It consists of (1) a perturbation module that samples new tokens to replace the raw ones in $Doc$ via LDP and (2) an extraction module that locally aligns the perturbed generation with the raw document.
  • Figure 3: The extraction module employs a smaller language model locally to extract text from the perturbed generation. It then reconstructs this text into an output that aligns with the raw document. We mark the text in green to indicate that it is identical in both the perturbed and extracted generations.
  • Figure 4: The number of tokens from the non-private and private generation of GPT4 using three mechanisms that belong to the Expected set.
  • Figure 5: The workflow of RANTEXT. It comprises two steps: (1) computing random adjacency lists and (2) sampling perturbed tokens via $\varepsilon$-LDP.
  • ...and 5 more figures

Theorems & Definitions (6)

  • Definition 1: $\varepsilon$-Local Differential Privacy kasiviswanathan2011can
  • Definition 2: Exponential Mechanism 4389483
  • Definition 3: Random Adjacent Embeddings
  • Definition 4: Random Adjacency List
  • proof : Proof of Theorem 1
  • proof : Proof of Theorem 2