Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applications

Chaoran Chen, Daodao Zhou, Yanfang Ye, Toby Jia-jun Li, Yaxing Yao

TL;DR

CLEAR addresses the privacy challenges of end-user LLM interactions by providing just-in-time, contextualized privacy policy analysis and risk generation. It combines a co-design study to derive design goals with a three-component system that detects sensitive input, extracts policy snippets, and generates risks, evaluated in ChatGPT and Gmail Gemini contexts. Results indicate improvements in usability and customer privacy awareness, with participants more cautious about sharing personal data and more likely to modify prompts. The work highlights AI's dual role in privacy: while introducing risks, it can empower users through transparent, context-aware privacy support and informed consent practices.

Abstract

The rise of end-user applications powered by large language models (LLMs), including both conversational interfaces and add-ons to existing graphical user interfaces (GUIs), introduces new privacy challenges. However, many users remain unaware of the risks. This paper explores methods to increase user awareness of privacy risks associated with LLMs in end-user applications. We conducted five co-design workshops to uncover user privacy concerns and their demand for contextual privacy information within LLMs. Based on these insights, we developed CLEAR (Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation), a just-in-time contextual assistant designed to help users identify sensitive information, summarize relevant privacy policies, and highlight potential risks when sharing information with LLMs. We evaluated the usability and usefulness of CLEAR across two example domains: ChatGPT and the Gemini plugin in Gmail. Our findings demonstrated that CLEAR is easy to use and improves users' understanding of data practices and privacy risks. We also discussed LLM's duality in posing and mitigating privacy risks, offering design and policy implications.

CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applications

TL;DR

CLEAR addresses the privacy challenges of end-user LLM interactions by providing just-in-time, contextualized privacy policy analysis and risk generation. It combines a co-design study to derive design goals with a three-component system that detects sensitive input, extracts policy snippets, and generates risks, evaluated in ChatGPT and Gmail Gemini contexts. Results indicate improvements in usability and customer privacy awareness, with participants more cautious about sharing personal data and more likely to modify prompts. The work highlights AI's dual role in privacy: while introducing risks, it can empower users through transparent, context-aware privacy support and informed consent practices.

Abstract

The rise of end-user applications powered by large language models (LLMs), including both conversational interfaces and add-ons to existing graphical user interfaces (GUIs), introduces new privacy challenges. However, many users remain unaware of the risks. This paper explores methods to increase user awareness of privacy risks associated with LLMs in end-user applications. We conducted five co-design workshops to uncover user privacy concerns and their demand for contextual privacy information within LLMs. Based on these insights, we developed CLEAR (Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation), a just-in-time contextual assistant designed to help users identify sensitive information, summarize relevant privacy policies, and highlight potential risks when sharing information with LLMs. We evaluated the usability and usefulness of CLEAR across two example domains: ChatGPT and the Gemini plugin in Gmail. Our findings demonstrated that CLEAR is easy to use and improves users' understanding of data practices and privacy risks. We also discussed LLM's duality in posing and mitigating privacy risks, offering design and policy implications.

Paper Structure

This paper contains 56 sections, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Privacy Risks of LLMs
  4. Existing Methods for Protecting User Privacy when using LLM-enabled Applications
  5. Contextual Privacy Policy
  6. Co-Design Workshops
  7. Participants
  8. Workshop Procedure
  9. Data Analysis
  10. Key Insights
  11. KI1: Concerns About Privacy Policies in AI Contexts.
  12. KI2: Expectations for Contextual, Simplified, and Focused Privacy Information
  13. KI3: Desire for Enhanced Data Protection and Increased Privacy Awareness
  14. KI4: Aspirations for AI to Enhance Privacy Control and Protection
  15. Design Goals
  16. ...and 41 more sections

Figures (4)

  • Figure 1: System flow of CLEAR
  • Figure 2: The user interface of CLEAR in the Gmail Context. In this example, a user attempts to use the Gemini plugin in Gmail to summarize an email containing sensitive information, such as an address and phone number. CLEAR detects and highlights the sensitive information in a pop-up(a). When the user clicks "Know more...", the pop-up expands to show relevant privacy policy excerpts from Gemini, along with potential privacy risks identified by the LLM(b). The black rectangles in the image are added to the screenshot to obfuscate the user's information from being disclosed in the figure.
  • Figure 3: Result of Post-study Questionnaire for the Case Study 1.
  • Figure 4: Result of Post-study Questionnaire for the Case Study 2.