Activation-Space Anchored Access Control for Multi-Class Permission Reasoning in Large Language Models

TL;DR

This work tackles the challenge of enforcing fine-grained, multi-class permission constraints in LLM-powered QA over knowledge bases. It introduces Activation-space Anchored Access Control (AAAC), a training-free framework that detects permission signals in mid-to-late activation layers and steers risky activations toward permission-specific anchors, thereby suppressing unauthorized content while preserving authorized utility. The method constructs per-permission anchors offline from small labeled samples and applies a risk-aware, multi-anchor steering mechanism online, guided by an Activation-Space Informativeness score and a two-threshold policy. Empirical results on three diverse LLMs and the new MultiPER-Enterprise benchmark show substantial reductions in permission violations (up to 86.5%) and prompt-based attack success (up to 90.7%), with modest inference overhead and robust performance across configurations. The work also provides analyses of layer selection, steering strength, and robustness, and discusses deployment considerations, limitations, and societal impact.

Abstract

Large language models (LLMs) are increasingly deployed over knowledge bases for efficient knowledge retrieval and question answering. However, LLMs can inadvertently answer beyond a user's permission scope, leaking sensitive content, thus making it difficult to deploy knowledge-base QA under fine-grained access control requirements. In this work, we identify a geometric regularity in intermediate activations: for the same query, representations induced by different permission scopes cluster distinctly and are readily separable. Building on this separability, we propose Activation-space Anchored Access Control (AAAC), a training-free framework for multi-class permission control. AAAC constructs an anchor bank, with one permission anchor per class, from a small offline sample set and requires no fine-tuning. At inference time, a multi-anchor steering mechanism redirects each query's activations toward the anchor-defined authorized region associated with the current user, thereby suppressing over-privileged generations by design. Finally, extensive experiments across three LLM families demonstrate that AAAC reduces permission violation rates by up to 86.5% and prompt-based attack success rates by 90.7%, while improving response usability with minor inference overhead compared to baselines.

Figures (4)

• Figure 1: Conceptual comparison between general LLM alignment and multi-class access control. General alignment uniformly restricts access to sensitive knowledge, whereas access-aware control selectively enables authorized responses based on permissions.
• Figure 2: Separable activation clusters across permission roles. t-SNE of last-token activations from a mid-layer of three LLMs under identical queries. Different colors indicate different permissions; stars mark per-permission activation anchors. Clear cluster separation indicates that permission semantics are reflected in the model’s intermediate geometry.
• Figure 3: Workflow of Activation-space Anchored Access Control (AAAC).
• Figure 4: Layer-wise composite permission score (ASI-Score). The strongest permission-related signals concentrate around layers 11--13 for Llama3-8B-Instruct, 16--18 for Gemma-7B-it, and 19--21 for Qwen3-8B, which we select as the AAAC control set $\mathcal{L}$.