Personalizing Agent Privacy Decisions via Logical Entailment

TL;DR

The paper tackles personalized privacy decisions for personal AI agents by grounding judgments in a user’s prior data-sharing decisions rather than relying on broad privacy norms. It introduces ARIEL, a neurosymbolic framework that uses LLM-generated ontologies and rule-based entailment to determine whether incoming data-sharing requests are entailed by prior judgments, escalating when needed to preserve user agency. Empirical evaluation on SPA and Education datasets shows ARIEL significantly reduces judgment errors (up to 39.1% for appropriate judgments) compared with purely LLM-based reasoning, with robust performance even for smaller models. This work demonstrates that combining structured logical entailment with neural reasoning yields auditable, personalized privacy decisions suitable for on-device deployment and real-world agent use.

Abstract

Personal language model-based agents are becoming more widespread for completing tasks on behalf of users; however, this raises serious privacy questions regarding whether these models will appropriately disclose user data. While prior work has evaluated language models on data-sharing scenarios based on general privacy norms, we focus on personalizing language models' privacy decisions, grounding their judgments directly in prior user privacy decisions. Our findings suggest that general privacy norms are insufficient for effective personalization of privacy decisions. Furthermore, we find that eliciting privacy judgments from the model through In-context Learning (ICL) is unreliable to due misalignment with the user's prior privacy judgments and opaque reasoning traces, which make it difficult for the user to interpret the reasoning behind the model's decisions. To address these limitations, we propose ARIEL (Agentic Reasoning with Individualized Entailment Logic), a framework that jointly leverages a language model and rule-based logic for structured data-sharing reasoning. ARIEL is based on formulating personalization of data sharing as an entailment, whether a prior user judgment on a data-sharing request implies the same judgment for an incoming request. Our experimental evaluations on advanced models and publicly-available datasets demonstrate that ARIEL can reduce the F1 score error by $\textbf{39.1%}$ over language model-based reasoning (ICL), demonstrating that ARIEL is effective at correctly judging requests where the user would approve data sharing. Overall, our findings suggest that combining LLMs with strict logical entailment is a highly effective strategy for enabling personalized privacy judgments for agents.

Figures (6)

• Figure 1: High-level overview of a personal agent completing a task on behalf of a user. A third party is requesting user data in order to complete the task. The agent must decide whether it is appropriate to share the user's information to fulfill the third-party's request. To assist with the agent's judgment, the agent is equipped with a knowledge base storing a history of prior user judgments on data-sharing requests.
• Figure 2: High-level overview of ARIEL, which can be broken down into three components. ① ARIEL generates an ontology for each user based on their prior privacy judgments from the knowledge base using the LLM. ② Once ARIEL receives an incoming request, it goes through each prior request in the user's knowledge base to determine if entailment holds. It uses the LLM and the generated ontologies to map the parameters in both requests to correspond levels in the ontologies. ③ A set of rules are applied to determine if entailment holds between each mapped prior requests and the mapped prior request. If the incoming request is not entailed by any prior requests, ARIEL escalates the incoming request to the user.
• Figure 3: Example of generated data type ontology from Gemini 2.5 Pro.
• Figure 4: Example requests from the SPA and Education Dataset that are provided to the LLMs in our evaluations.
• Figure 5: Ablation study on the number of prior requests with user judgments contained in the user's knowledge base $D_u$. We evaluate ICL w (Undet) and ARIEL on the SPA dataset with Gemini 2.5 Pro. We report the F1-score for appropriate $F1_{\text{A}}$ and inappropriate $F1_{\text{I}}$ class, and the total number of appropriateness judgments (Sup). We find that ARIEL is more robust to varying the number of prior requests compared to ICL w (Undet)

Theorems & Definitions (5)

• Example 3.1
• Definition 3.1: Request
• Example 4.1
• Definition 4.1: Ontological Relationship
• Definition 4.2: Entailment