Adanonymizer: Interactively Navigating and Balancing the Duality of Privacy and Output Performance in Human-LLM Interaction

TL;DR

The paper tackles the problem of balancing privacy protection and output performance in human-LLM interactions, revealing a privacy paradox where users frequently disclose sensitive data despite risk awareness. It introduces Adanonymizer, a local plug-in with a 2D color palette that lets users collaboratively control privacy and utility through a prompt-based, pseudo-anonymization pipeline. Study 1 quantifies privacy risks across 14 PI categories and demonstrates weak correlation between privacy risk and model performance, informing the design of the interface. Study 2 shows Adanonymizer reduces modification time and yields higher perceived performance and satisfaction than ablations and DP baselines, across multiple scenarios, highlighting the practicality and user value of interactive privacy management. Overall, the work provides a human-centric framework for privacy-utility trade-offs in text-based H-LLM interactions and suggests design directions for future privacy-aware interfaces.

Abstract

Current Large Language Models (LLMs) cannot support users to precisely balance privacy protection and output performance during individual consultations. We introduce Adanonymizer, an anonymization plug-in that allows users to control this balance by navigating a trade-off curve. A survey (N=221) revealed a privacy paradox, where users frequently disclosed sensitive information despite acknowledging privacy risks. The study further demonstrated that privacy risks were not significantly correlated with model output performance, highlighting the potential to navigate this trade-off. Adanonymizer normalizes privacy and utility ratings by type and automates the pseudonymization of sensitive terms based on user preferences, significantly reducing user effort. Its 2D color palette interface visualizes the privacy-utility trade-off, allowing users to adjust the balance by manipulating a point. An evaluation (N=36) compared Adanonymizer with ablation methods and differential privacy techniques, where Adanonymizer significantly reduced modification time, achieved better perceived model performance and overall user preference.

Figures (9)

• Figure 1: The illustration of private information's appearance frequency, 7: most frequent, 1: least frequent. The errorbar indicated one standard deviation.
• Figure 2: The trade-off between privacy risk (1: least private, 7: most private) and usage frequency (1: least frequent, 7: most frequent). Different information is aggregated into different categories with different colors. We used light blue shading to highlight the envelope, allowing for observation of the overall trend.
• Figure 3: The trade-off between privacy (1: least private, 7: most private) and the influence on model's performance (1: least influential, 7: most influential) for different information. Different information is aggregated into different categories with different colors.
• Figure 4: Different layout designs of the 2D palette.
• Figure 5: The candidate interface designs for privacy-utility balancing.