Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Privacy Control in Conversational LLM Platforms: A Walkthrough Study

Zhuoyang Li, Yanlai Wu, Yao Li, Xinning Gui, Yuhan Luo

TL;DR

This paper investigates how six consumer-facing conversational LLM platforms implement interface-level privacy controls, revealing emergent data units such as chat history, memory, and customized objects that are shaped by natural-language controls. An expert-driven application walkthrough documents governance, control options, and execution modes across platforms, highlighting co-ownership implications in multi-user contexts. The study finds NL-based control offers intuitive interaction but introduces ambiguity and notes that most controls are retrospective with limited proactive governance, though some platforms provide proactive features like temporary chat and auto-deletion. These insights point to design directions for scalable NL privacy controls and advocate for governance frameworks addressing co-owned data, with practical implications for developers and policymakers.

Abstract

Large language models (LLMs) are increasingly integrated into daily life through conversational interfaces, processing user data via natural language inputs and exhibiting advanced reasoning capabilities, which raises new concerns about user control over privacy. While much research has focused on potential privacy risks, less attention has been paid to the data control mechanisms these platforms provide. This study examines six conversational LLM platforms, analyzing how they define and implement features for users to access, edit, delete, and share data. Our analysis reveals an emerging paradigm of data control in conversational LLM platforms, where user data is generated and derived through interaction itself, natural language enables flexible yet often ambiguous control, and multi-user interactions with shared data raise questions of co-ownership and governance. Based on these findings, we offer practical insights for platform developers, policymakers, and researchers to design more effective and usable privacy controls in LLM-powered conversational interactions.

Privacy Control in Conversational LLM Platforms: A Walkthrough Study

TL;DR

This paper investigates how six consumer-facing conversational LLM platforms implement interface-level privacy controls, revealing emergent data units such as chat history, memory, and customized objects that are shaped by natural-language controls. An expert-driven application walkthrough documents governance, control options, and execution modes across platforms, highlighting co-ownership implications in multi-user contexts. The study finds NL-based control offers intuitive interaction but introduces ambiguity and notes that most controls are retrospective with limited proactive governance, though some platforms provide proactive features like temporary chat and auto-deletion. These insights point to design directions for scalable NL privacy controls and advocate for governance frameworks addressing co-owned data, with practical implications for developers and policymakers.

Abstract

Large language models (LLMs) are increasingly integrated into daily life through conversational interfaces, processing user data via natural language inputs and exhibiting advanced reasoning capabilities, which raises new concerns about user control over privacy. While much research has focused on potential privacy risks, less attention has been paid to the data control mechanisms these platforms provide. This study examines six conversational LLM platforms, analyzing how they define and implement features for users to access, edit, delete, and share data. Our analysis reveals an emerging paradigm of data control in conversational LLM platforms, where user data is generated and derived through interaction itself, natural language enables flexible yet often ambiguous control, and multi-user interactions with shared data raise questions of co-ownership and governance. Based on these findings, we offer practical insights for platform developers, policymakers, and researchers to design more effective and usable privacy controls in LLM-powered conversational interactions.
Paper Structure (43 sections, 11 figures, 5 tables)

This paper contains 43 sections, 11 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. In-platform Privacy Controls
  4. Privacy in Conversational LLM Platforms
  5. Method
  6. Methodological Considerations
  7. Application Walkthrough as an Expert-driven Approach
  8. Analytic Lens Within Temporal Limits
  9. Researchers' Positionality
  10. Platform Identification and Selection
  11. Searching LLMs
  12. Screening LLMs and Tracing LLM Platforms
  13. Screening Conversational LLM Platforms
  14. Walkthrough Procedure
  15. Analysis of Expected Environment of Use
  16. ...and 28 more sections

Figures (11)

  • Figure 1: The three-stage data collection process.
  • Figure 2: Interface examples of ChatGPT (free plan). Conversational User Interface (CUI) is the main interface where users interact with ChatGPT. Memory Widget appears when a user input triggers the memory-related feature, where users can view memory snippets and access the memory portal. Side Panel is typically located next to the chat window, which provides access to: chat history, organized by chat sessions; customization GPTs created by other users and Customized GPTs created by the user; customization Store for discovering and interacting with customized CAs created by other users. Settings Page is a centralized hub for managing various user preferences. Shared Links Portal provides access to and control on shared conversations. Customization Settings allows users to input descriptive information to customize their ChatGPT. Memory Portal is a space where users can manage all stored memory snippets.
  • Figure 3: In Gemini, the data units of chat history include a message, a conversational round, and a chat session. When a message generated by Gemini is based on "saved info" (i.e., memory), a notification appears indicating which memory snippet was referenced.
  • Figure 4: The "Gemini Apps Activity" page, where users can manage their interaction history with Gemini. Users can choose to "Turn off" or "Turn off and delete activity". Turning off activity prevents future chats from appearing in the activity log and from being used to train the models. Choosing "Turn off and delete activity" also deletes all chat history. Users can manually delete individual conversational rounds or multiple rounds within "Last hour," "Last day," "Always," or a "Custom range." Users can enable auto-deletion within the Gemini app, with options for "3 months," "18 months" (default), "36 months," or "Not auto-delete activity." Chat history is grouped under "Item Details," which represents a conversational round containing one user input and one Gemini output.
  • Figure 5: Gemini's memory snippets and Saved Info portal. Using a prompt such as "Remember ...," users can instruct Gemini to save information as memory. Saved information is displayed in the "Info that you asked Gemini to save" page, where the memory is organized by snippets (pieces of information). Users can access, edit, or delete individual snippets, and can also delete all snippets from this interface.
  • ...and 6 more figures