Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DP-GTR: Differentially Private Prompt Protection via Group Text Rewriting

Mingchen Li, Heng Fan, Song Fu, Junhua Ding, Yunhe Feng

TL;DR

Prompt privacy in LLM services is a growing concern, and existing DP approaches struggle to balance privacy with utility without heavy training or coarse control. DP-GTR introduces a three-stage, DP-enabled group text rewriting framework that leverages in-context learning to optimize utility while identifying and suppressing consensus private keywords, bridging local and global DP. The method is plug-in compatible with existing paraphrasers and demonstrates superior privacy-utility trade-offs on open- and closed-answer QA tasks (e.g., DocVQA and CSQA) under realistic evaluation. This work offers a practical, scalable approach to protecting user prompts in deployment, with strong robustness to adversarial settings and clear guidance for future LDP-enabled prompting pipelines.

Abstract

Prompt privacy is crucial, especially when using online large language models (LLMs), due to the sensitive information often contained within prompts. While LLMs can enhance prompt privacy through text rewriting, existing methods primarily focus on document-level rewriting, neglecting the rich, multi-granular representations of text. This limitation restricts LLM utilization to specific tasks, overlooking their generalization and in-context learning capabilities, thus hindering practical application. To address this gap, we introduce DP-GTR, a novel three-stage framework that leverages local differential privacy (DP) and the composition theorem via group text rewriting. DP-GTR is the first framework to integrate both document-level and word-level information while exploiting in-context learning to simultaneously improve privacy and utility, effectively bridging local and global DP mechanisms at the individual data point level. Experiments on CommonSense QA and DocVQA demonstrate that DP-GTR outperforms existing approaches, achieving a superior privacy-utility trade-off. Furthermore, our framework is compatible with existing rewriting techniques, serving as a plug-in to enhance privacy protection. Our code is publicly available at github.com/ResponsibleAILab/DP-GTR.

DP-GTR: Differentially Private Prompt Protection via Group Text Rewriting

TL;DR

Prompt privacy in LLM services is a growing concern, and existing DP approaches struggle to balance privacy with utility without heavy training or coarse control. DP-GTR introduces a three-stage, DP-enabled group text rewriting framework that leverages in-context learning to optimize utility while identifying and suppressing consensus private keywords, bridging local and global DP. The method is plug-in compatible with existing paraphrasers and demonstrates superior privacy-utility trade-offs on open- and closed-answer QA tasks (e.g., DocVQA and CSQA) under realistic evaluation. This work offers a practical, scalable approach to protecting user prompts in deployment, with strong robustness to adversarial settings and clear guidance for future LDP-enabled prompting pipelines.

Abstract

Prompt privacy is crucial, especially when using online large language models (LLMs), due to the sensitive information often contained within prompts. While LLMs can enhance prompt privacy through text rewriting, existing methods primarily focus on document-level rewriting, neglecting the rich, multi-granular representations of text. This limitation restricts LLM utilization to specific tasks, overlooking their generalization and in-context learning capabilities, thus hindering practical application. To address this gap, we introduce DP-GTR, a novel three-stage framework that leverages local differential privacy (DP) and the composition theorem via group text rewriting. DP-GTR is the first framework to integrate both document-level and word-level information while exploiting in-context learning to simultaneously improve privacy and utility, effectively bridging local and global DP mechanisms at the individual data point level. Experiments on CommonSense QA and DocVQA demonstrate that DP-GTR outperforms existing approaches, achieving a superior privacy-utility trade-off. Furthermore, our framework is compatible with existing rewriting techniques, serving as a plug-in to enhance privacy protection. Our code is publicly available at github.com/ResponsibleAILab/DP-GTR.

Paper Structure

This paper contains 34 sections, 5 equations, 3 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Pure Differential Privacy (DP)
  5. Local Differential Privacy
  6. Metric Differential Privacy
  7. Exponential Mechanism
  8. Composition Property
  9. Post-Processing Property
  10. DP-Guaranteed Paraphrasing
  11. DP-GTR
  12. DP-GTR Framework Overview
  13. Stage-1: Group Text Rewriting
  14. Stage-2: Utility and Privacy Control
  15. One-shot in-context learning for utility
  16. ...and 19 more sections

Figures (3)

  • Figure 1: DP-GTR: A three-stage pipeline for Differentially Private prompt protection via Group Text Rewriting (GTR). Stage-1 generates $n$ paraphrases of the original prompt using a DP paraphrasing mechanism. Stage-2 identifies the lowest-perplexity prompt as the ICL exemplar and aggregates word counts to release $k$ private keywords. Stage-3 integrates these private keywords and the ICL exemplar into a prompt template for submission to the LLM, producing the final, differentially private prompt.
  • Figure 2: Privacy-utility trade-off for baselines and DP-GTR on open-answer PFL-DocVQA (VQA) dataset. The left column presents GPT-3.5 results, and the right column shows Llama-3.1-8B results. Refer to the x-axis label for specific measurement metrics.
  • Figure 3: Privacy-utility trade-off for baselines and DP-GTR on close-answer Commonsense QA (CSQA) dataset. The left column presents GPT-3.5 results, and the right column shows Llama-3.1-8B results. Refer to the x-axis label for specific measurement metrics.