The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Prompts

TL;DR

ProSan is an end-to-end prompt privacy protection framework that can produce anonymized prompts with contextual privacy removed while maintaining task usability and human readability and can also be seamlessly integrated into the online LLM service pipeline.

Abstract

The rapid adoption of online chatbots represents a significant advancement in artificial intelligence. However, this convenience brings considerable privacy concerns, as prompts can inadvertently contain sensitive information exposed to large language models (LLMs). Limited by high computational costs, reduced task usability, and excessive system modifications, previous works based on local deployment, embedding perturbation, and homomorphic encryption are inapplicable to online prompt-based LLM applications. To address these issues, this paper introduces Prompt Privacy Sanitizer (i.e., ProSan), an end-to-end prompt privacy protection framework that can produce anonymized prompts with contextual privacy removed while maintaining task usability and human readability. It can also be seamlessly integrated into the online LLM service pipeline. To achieve high usability and dynamic anonymity, ProSan flexibly adjusts its protection targets and strength based on the importance of the words and the privacy leakage risk of the prompts. Additionally, ProSan is capable of adapting to diverse computational resource conditions, ensuring privacy protection even for mobile devices with limited computing power. Our experiments demonstrate that ProSan effectively removes private information across various tasks, including question answering, text summarization, and code generation, with minimal reduction in task performance.

Figures (10)

• Figure 1: Prompt Privacy Sanitizer illustration: Prompt Privacy Sanitizer measures the importance and privacy of words in the prompt and selectively replaces them. It can generate anonymized prompts for various tasks while maintaining high usability and dynamic anonymity.
• Figure 2: The kernel density estimate plot of the importance scores for stems and options in multiple-choice questions from the MedQA dataset.
• Figure 3: A visualization of word importance within a prompt. Darker colors indicate greater importance.
• Figure 4: The kernel density estimate plot of the self-information for named entities and non-named entities from the PII-masking-43k dataset.
• Figure 5: A visualization of privacy leak risks for same category and same word. Darker colors indicate higher privacy leakage risk.