PrivacyReasoner: Can LLM Emulate a Human-like Privacy Mind?

TL;DR

PrivacyReasoner introduces a cognitively informed AI agent that emulates individual human privacy minds by building a personalized privacy memory from historical user comments and post context, then activating context-relevant orientations to generate synthetic responses to new privacy events. Grounded in Contextual Integrity and the Antecedents–Privacy Concerns–Outcomes (APCO) framework, it decouples stable privacy dispositions from context-driven activation, using memory distillation and contextual filtering, and evaluates reasoning faithfulness with an LLM-as-a-judge calibrated to a 14-category privacy taxonomy. Empirical results on Hacker News data show PRA outperforms baselines and generalizes across domains and users, with a CSAM case study illustrating its ability to surface nuanced, human-like objections. The work offers practical value for design prototyping and policy auditing, while outlining limitations related to data representativeness, judge biases, and single-mode analysis, and points to future avenues in richer cognition and multi-agent societal simulations.

Abstract

This paper introduces PRA, an AI-agent design for simulating how individual users form privacy concerns in response to real-world news. Moving beyond population-level sentiment analysis, PRA integrates privacy and cognitive theories to simulate user-specific privacy reasoning grounded in personal comment histories and contextual cues. The agent reconstructs each user's "privacy mind", dynamically activates relevant privacy memory through a contextual filter that emulates bounded rationality, and generates synthetic comments reflecting how that user would likely respond to new privacy scenarios. A complementary LLM-as-a-Judge evaluator, calibrated against an established privacy concern taxonomy, quantifies the faithfulness of generated reasoning. Experiments on real-world Hacker News discussions show that \PRA outperforms baseline agents in privacy concern prediction and captures transferable reasoning patterns across domains including AI, e-commerce, and healthcare.

Figures (3)

• Figure 1: Overview of the $\mathrm{PrivacyReasoner}$ workflow, an agent designed to mimic how individuals react to privacy-related events. These events are simulated using discussions from Hacker News, a technical forum. The agent constructs its privacy memory from two components: (1) the user’s historical comments, which serve as the raw substrate for building the agent’s profile, and (2) the post context, which includes metadata such as the post title, body text, and post initiator ID. Conditioned on this memory, the agent employs a contextual filter to selectively activate relevant privacy orientations and generate a synthetic comment simulating the user’s likely response. Both the synthetic and real comments are then evaluated by a Privacy Concern Judge, which leverages an established privacy concern taxonomy and human-annotated interview data to assess concern-level alignment.
• Figure 2: Effect of privacy memory size on privacy concern prediction. Increasing the number of historical user comments available to both $\mathrm{PrivacyReasoner}$ and summary-based variant. We report macro F1-scores as the evaluation metric, as well as the standard deviation.
• Figure 3: User transfer performance of the $\mathrm{PrivacyReasoner}$ agent constructed with varying amounts of user comment history.