Can LLMs Make (Personalized) Access Control Decisions?

TL;DR

The paper tackles the cognitive burden of making runtime access-control decisions by leveraging large language models guided by brief natural-language privacy statements. It conducts a large user study to build a dataset of 307 privacy statements and 14,682 access-control decisions, and evaluates both generic and personalized LLM configurations against user decisions and expert recommendations. Findings show that generic LLMs align with general human judgment and security best practices, while personalization can meaningfully improve agreement for some users but may degrade for others, and can even yield safer decisions in aggregate when coupled with explanations. The work provides design and risk considerations for deploying natural-language-based access control that balances personalization, security, and usability, and suggests configurable thresholds and iterative preference refinement as practical mechanisms to manage accuracy and user burden.

Abstract

Precise access control decisions are crucial to the security of both traditional applications and emerging agent-based systems. Typically, these decisions are made by users during app installation or at runtime. Due to the increasing complexity and automation of systems, making these access control decisions can add a significant cognitive load on users, often overloading them and leading to suboptimal or even arbitrary access control decisions. To address this problem, we propose to leverage the processing and reasoning capabilities of large language models (LLMs) to make dynamic, context-aware decisions aligned with the user's security preferences. For this purpose, we conducted a user study, which resulted in a dataset of 307 natural-language privacy statements and 14,682 access control decisions made by users. We then compare these decisions against those made by two versions of LLMs: a general and a personalized one, for which we also gathered user feedback on 1,446 of its decisions. Our results show that in general, LLMs can reflect users' preferences well, achieving up to 86\% accuracy when compared to the decision made by the majority of users. Our study also reveals a crucial trade-off in personalizing such a system: while providing user-specific privacy preferences to the LLM generally improves agreement with individual user decisions, adhering to those preferences can also violate some security best practices. Based on our findings, we discuss design and risk considerations for implementing a practical natural-language-based access control system that balances personalization, security, and utility.

Figures (8)

• Figure 1: Summary of this paper. We collect users' access control stance through a brief statement, which we use to get personalized access control decisions from an LLM. We measure the agreement between user and LLM decisions, and the cases where the LLM made a preferable choice compared to users, either as reported through feedback or through preventing security violations.
• Figure 2: LLM-Based access control decision making.
• Figure 3: Confusion matrices of generic and personalized LLM decisions compared to individual user decisions.
• Figure 4: Per-user agreement of generic (G) vs. personalized (P) decisions and the difference per user (P-G).
• Figure 5: Privacy Statement length versus $P_{\text{4o}}$ agreement.