Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EmojiPrompt: Generative Prompt Obfuscation for Privacy-Preserving Communication with Cloud-based LLMs

Sam Lin, Wenyue Hua, Zhenting Wang, Mingyu Jin, Lizhou Fan, Yongfeng Zhang

TL;DR

EmojiPrompt tackles privacy concerns in cloud-based LLMs by transforming private data in prompts into non-natural language forms via a dual-LLM pipeline. It introduces atomic-level obfuscation with reusable and non-reusable modes and enforces privacy guarantees through semantic-alignment and $ ext{LDP}$ post-sampling constraints, enabling fully cloud-based deployment without local models. Empirically, it achieves comparable or superior task performance to unobfuscated prompts across eight datasets while demonstrating stronger resistance to LLM- and human-based inference attacks, and it remains effective across multiple languages. The approach supports automatic prompt optimization (APE/OPRO) and shows practical potential for privacy-preserving AI services, albeit with limitations related to vocabulary and potential hallucinations in obfuscation outputs.

Abstract

Cloud-based Large Language Models (LLMs) such as ChatGPT have become increasingly integral to daily operations. Nevertheless, they also introduce privacy concerns: firstly, numerous studies underscore the risks to user privacy posed by jailbreaking cloud-based LLMs; secondly, the LLM service providers have access to all user data, which deters individuals from confidently utilizing such services. To address such concerns, we propose a simple yet effective paradigm, EmojiPrompt, to protect user privacy. At its core, EmojiPrompt performs generative transformation, obfuscating private data within prompts with linguistic and non-linguistic elements before submitting them to cloud-based LLMs. We evaluate EmojiPrompt's performance across 8 datasets from various domains. We also propose simulated inference attacks to assess EmojiPrompt's ability to preserve user privacy. The results demonstrate that EmojiPrompt effectively obfuscates user private data, while largely maintaining, or even enhancing, performances compared to the unobfuscated version. Furthermore, EmojiPrompt's atomic-level obfuscation allows it to function exclusively with cloud-based LLMs. For source code, please refer to: https://github.com/agiresearch/EmojiCrypt.

EmojiPrompt: Generative Prompt Obfuscation for Privacy-Preserving Communication with Cloud-based LLMs

TL;DR

EmojiPrompt tackles privacy concerns in cloud-based LLMs by transforming private data in prompts into non-natural language forms via a dual-LLM pipeline. It introduces atomic-level obfuscation with reusable and non-reusable modes and enforces privacy guarantees through semantic-alignment and post-sampling constraints, enabling fully cloud-based deployment without local models. Empirically, it achieves comparable or superior task performance to unobfuscated prompts across eight datasets while demonstrating stronger resistance to LLM- and human-based inference attacks, and it remains effective across multiple languages. The approach supports automatic prompt optimization (APE/OPRO) and shows practical potential for privacy-preserving AI services, albeit with limitations related to vocabulary and potential hallucinations in obfuscation outputs.

Abstract

Cloud-based Large Language Models (LLMs) such as ChatGPT have become increasingly integral to daily operations. Nevertheless, they also introduce privacy concerns: firstly, numerous studies underscore the risks to user privacy posed by jailbreaking cloud-based LLMs; secondly, the LLM service providers have access to all user data, which deters individuals from confidently utilizing such services. To address such concerns, we propose a simple yet effective paradigm, EmojiPrompt, to protect user privacy. At its core, EmojiPrompt performs generative transformation, obfuscating private data within prompts with linguistic and non-linguistic elements before submitting them to cloud-based LLMs. We evaluate EmojiPrompt's performance across 8 datasets from various domains. We also propose simulated inference attacks to assess EmojiPrompt's ability to preserve user privacy. The results demonstrate that EmojiPrompt effectively obfuscates user private data, while largely maintaining, or even enhancing, performances compared to the unobfuscated version. Furthermore, EmojiPrompt's atomic-level obfuscation allows it to function exclusively with cloud-based LLMs. For source code, please refer to: https://github.com/agiresearch/EmojiCrypt.
Paper Structure (40 sections, 8 equations, 4 figures, 12 tables)

This paper contains 40 sections, 8 equations, 4 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem Definition
  5. Atomic-level Obfuscation
  6. Reusable Obfuscation
  7. Non-Reusable Obfuscation
  8. Theoretical Grounding
  9. Semantic alignment constraint
  10. LDP Post-sampling constraint
  11. Obfuscation Rationale
  12. Experiments
  13. Experiment Setup
  14. Dataset and Metric
  15. Modeling Setup
  16. ...and 25 more sections

Figures (4)

  • Figure 1: Illustration of EmojiPrompt for preserving user privacy in LLM-powered personalized recommender systems, using LLM$_{\mathcal{O}}$ to transform product titles in user behavior history into emoji sequences. The LLM$_{\mathcal{I}}$ then processes the obfuscated prompt to infer and generate relevant product recommendations.
  • Figure 2: Non-Reusable Obfuscation and Rationale on a movie review
  • Figure 3: Obfuscation Rationale on Beauty Products.
  • Figure 4: Illustration of EmojiPrompt for preserving user privacy on tabular data.

Theorems & Definitions (3)

  • Definition 3.1: Text-based adjacency
  • Definition 3.2: Semantic Alignment
  • Definition 3.3: $\epsilon$-LDP