Exploring Large Language Models for Access Control Policy Synthesis and Summarization

TL;DR

This work investigates the use of Large Language Models for access control policy synthesis and summarization in cloud environments, identifying the Verifiable Synthesis Paradox where precise guarantees conflict with natural-language inputs. It systematically evaluates multiple LLMs on policy comprehension and reconstruction and introduces PolicySummarizer, a semantic-based request summarization framework that encodes policies as automata, extracts regular expressions, and validates them with model counting. The study finds that reasoning LLMs best achieve semantic equivalence but still exhibit non-trivial error rates, while LLMs excel at explanations but require symbolic methods for reliable verification. PolicySummarizer demonstrates substantial gains in producing concise, interpretable summaries of allowed requests and policy differences, outperforming SMT-based baselines by a significant margin and enabling more effective human-in-the-loop verification. The results suggest a promising path toward hybrid systems that combine LLM-based interpretation with formal methods for robust access control policy analysis and synthesis.

Abstract

Cloud computing is ubiquitous, with a growing number of services being hosted on the cloud every day. Typical cloud compute systems allow administrators to write policies implementing access control rules which specify how access to private data is governed. These policies must be manually written, and due to their complexity can often be error prone. Moreover, existing policies often implement complex access control specifications and thus can be difficult to precisely analyze in determining their behavior works exactly as intended. Recently, Large Language Models (LLMs) have shown great success in automated code synthesis and summarization. Given this success, they could potentially be used for automatically generating access control policies or aid in understanding existing policies. In this paper, we explore the effectiveness of LLMs for access control policy synthesis and summarization. Specifically, we first investigate diverse LLMs for access control policy synthesis, finding that: although LLMs can effectively generate syntactically correct policies, they have permissiveness issues, generating policies equivalent to the given specification 45.8% of the time for non-reasoning LLMs, and 93.7% of the time for reasoning LLMs. We then investigate how LLMs can be used to analyze policies by introducing a novel semantic-based request summarization approach which leverages LLMs to generate a precise characterization of the requests allowed by a policy. Our results show that while there are significant hurdles in leveraging LLMs for automated policy generation, LLMs show promising results when combined with symbolic approaches in analyzing existing policies.

Figures (5)

• Figure 1: LLM Policy Comprehension and Reconstruction results.
• Figure 2: Comprehensive Policy Comprehension Assessment.
• Figure 4: Projected DFA representing allowed resources from the motivating example in Figure \ref{['fig:motivation']}. For visual purposes, when multiple transitions exist between two states the transitions are grouped together into one.
• Figure 6: Results of the resource characterizations for original policies in the Quacky dataset.
• Figure 7: Comparison of average similarity measure based on number of strings used during generalization through string enumeration along with baseline performance for the 1000 string scenario