Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DePrompt: Desensitization and Evaluation of Personal Identifiable Information in Large Language Model Prompts

Xiongtao Sun, Gan Liu, Zhipeng He, Hui Li, Xiaoguang Li

TL;DR

DePrompt tackles prompt-based PII leakage in LLM workflows by introducing a local desensitization framework that combines LLM fine-tuning, semantic/privacy entity extraction, and adversarial generative desensitization to disrupt links between identifiers and privacy attributes while preserving semantic content. It defines prompt-centric anonymization attributes—linkability, semanticity, and uncertainty—and provides a dedicated privacy-utility evaluation scheme, showing improvements over traditional anonymization methods. Empirical results across medical, daily, and financial prompts indicate an average PII recognition accuracy of 95.95% and favorable privacy-utility tradeoffs, with DePrompt achieving strong scene recognition and private-entity extraction performance. The approach is adaptable to various text-utility-dependent scenarios and offers a practical path toward safer deployment of LLMs in privacy-sensitive applications.

Abstract

Prompt serves as a crucial link in interacting with large language models (LLMs), widely impacting the accuracy and interpretability of model outputs. However, acquiring accurate and high-quality responses necessitates precise prompts, which inevitably pose significant risks of personal identifiable information (PII) leakage. Therefore, this paper proposes DePrompt, a desensitization protection and effectiveness evaluation framework for prompt, enabling users to safely and transparently utilize LLMs. Specifically, by leveraging large model fine-tuning techniques as the underlying privacy protection method, we integrate contextual attributes to define privacy types, achieving high-precision PII entity identification. Additionally, through the analysis of key features in prompt desensitization scenarios, we devise adversarial generative desensitization methods that retain important semantic content while disrupting the link between identifiers and privacy attributes. Furthermore, we present utility evaluation metrics for prompt to better gauge and balance privacy and usability. Our framework is adaptable to prompts and can be extended to text usability-dependent scenarios. Through comparison with benchmarks and other model methods, experimental evaluations demonstrate that our desensitized prompt exhibit superior privacy protection utility and model inference results.

DePrompt: Desensitization and Evaluation of Personal Identifiable Information in Large Language Model Prompts

TL;DR

DePrompt tackles prompt-based PII leakage in LLM workflows by introducing a local desensitization framework that combines LLM fine-tuning, semantic/privacy entity extraction, and adversarial generative desensitization to disrupt links between identifiers and privacy attributes while preserving semantic content. It defines prompt-centric anonymization attributes—linkability, semanticity, and uncertainty—and provides a dedicated privacy-utility evaluation scheme, showing improvements over traditional anonymization methods. Empirical results across medical, daily, and financial prompts indicate an average PII recognition accuracy of 95.95% and favorable privacy-utility tradeoffs, with DePrompt achieving strong scene recognition and private-entity extraction performance. The approach is adaptable to various text-utility-dependent scenarios and offers a practical path toward safer deployment of LLMs in privacy-sensitive applications.

Abstract

Prompt serves as a crucial link in interacting with large language models (LLMs), widely impacting the accuracy and interpretability of model outputs. However, acquiring accurate and high-quality responses necessitates precise prompts, which inevitably pose significant risks of personal identifiable information (PII) leakage. Therefore, this paper proposes DePrompt, a desensitization protection and effectiveness evaluation framework for prompt, enabling users to safely and transparently utilize LLMs. Specifically, by leveraging large model fine-tuning techniques as the underlying privacy protection method, we integrate contextual attributes to define privacy types, achieving high-precision PII entity identification. Additionally, through the analysis of key features in prompt desensitization scenarios, we devise adversarial generative desensitization methods that retain important semantic content while disrupting the link between identifiers and privacy attributes. Furthermore, we present utility evaluation metrics for prompt to better gauge and balance privacy and usability. Our framework is adaptable to prompts and can be extended to text usability-dependent scenarios. Through comparison with benchmarks and other model methods, experimental evaluations demonstrate that our desensitized prompt exhibit superior privacy protection utility and model inference results.
Paper Structure (42 sections, 2 equations, 8 figures, 3 tables, 3 algorithms)

This paper contains 42 sections, 2 equations, 8 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related works
  3. Privacy-preserving data publishing
  4. Natural language processing
  5. Evaluation metrics
  6. System and threat models
  7. System model
  8. Threat model
  9. Preliminaries
  10. Anonymization Key Terms
  11. Prompt-based anonymization attributes
  12. Linkability
  13. Semanticity
  14. Uncertainty
  15. DePrompt
  16. ...and 27 more sections

Figures (8)

  • Figure 1: Public prompt dataset PII leakage situation.
  • Figure 2: The system model of our framework.
  • Figure 3: Prompt workflows and sources of threats.
  • Figure 4: An example prompt in the complete workflow of DePrompt
  • Figure 5: Specific formatting of prompts in fine-tuning.
  • ...and 3 more figures