PRIV-QA: Privacy-Preserving Question Answering for Cloud Large Language Models

TL;DR

This work tackles privacy risks in cloud-based QA by proposing PRIV-QA, a two-module framework that de-identifies user queries (Hide Module) and recovers them after cloud-model responses (Recover Module). It introduces SensitiveQA, the first bilingual general privacy QA dataset with 57k interactions to evaluate privacy protection and QA quality. Experiments demonstrate high precision/recall for sensitive-information detection (up to 91.36/89.40 in English) and strong extraction-defense rates, while maintaining competitive recovery quality (BLEU/METEOR/ROUGE) with manageable time overhead. The approach offers a practical privacy-utility trade-off for real-world cloud LLM usage, with code and data publicly available.

Abstract

The rapid development of large language models (LLMs) is redefining the landscape of human-computer interaction, and their integration into various user-service applications is becoming increasingly prevalent. However, transmitting user data to cloud-based LLMs presents significant risks of data breaches and unauthorized access to personal identification information. In this paper, we propose a privacy preservation pipeline for protecting privacy and sensitive information during interactions between users and LLMs in practical LLM usage scenarios. We construct SensitiveQA, the first privacy open-ended question-answering dataset. It comprises 57k interactions in Chinese and English, encompassing a diverse range of user-sensitive information within the conversations. Our proposed solution employs a multi-stage strategy aimed at preemptively securing user information while simultaneously preserving the response quality of cloud-based LLMs. Experimental validation underscores our method's efficacy in balancing privacy protection with maintaining robust interaction quality. The code and dataset are available at https://github.com/ligw1998/PRIV-QA.

Figures (7)

• Figure 1: When using cloud-based LLMs, users must send queries to providers who can access their private data. However, by integrating our plug-and-play PRIV-QA framework, we can maximize privacy protection while ensure high-quality responses.
• Figure 2: Workflow of our privacy-preserving framework. For a user query, the hide module performs sensitive information detection and substitution. Then, we obfuscate the non-privacy text excluding certain important words. The recover module obtains the LLM responses and restores the privacy information in the text.
• Figure 3: An example of tri-level classification to a given user query.
• Figure 4: Final response quality results for each method compared to ground truth. GPT-4o is selected as the win-tie-lose judge. We report the win&tie rate and lose rate.
• Figure 5: Trade-off between query protection and final response quality. The x-axis refers to the EDR (Extraction Defense Rate), and the y-axis refers to the win&tie rate against ground truth judged by GPT-4o. Top-right methods perform best.