AgentGuardian: Learning Access Control Policies to Govern AI Agent Behavior

TL;DR

AgentGuardian tackles the safety of autonomous AI agents by learning context-aware access-control policies that govern tool usage and multi-step workflows. It builds a CFG $G=(V,E)$ from benign execution traces and derives per-tool ABAC-like policies by embedding inputs and attributes, generating rules $R_{T,k}$ and an overall policy ACP that combines CFG conformity with input constraints. Enforcement is performed in real time via a lightweight Policy Enforcer integrated with LiteLLM, enabling continuous monitoring and rapid intervention. In experiments on Knowledge Assistant and IT Support agents, AgentGuardian detected most malicious or misleading inputs while preserving normal functionality, and its control-flow governance also mitigated hallucination-driven errors. The approach provides a scalable, auditable governance layer that improves safety and reliability of AI agents in real-world deployments.

Abstract

Artificial intelligence (AI) agents are increasingly used in a variety of domains to automate tasks, interact with users, and make decisions based on data inputs. Ensuring that AI agents perform only authorized actions and handle inputs appropriately is essential for maintaining system integrity and preventing misuse. In this study, we introduce the AgentGuardian, a novel security framework that governs and protects AI agent operations by enforcing context-aware access-control policies. During a controlled staging phase, the framework monitors execution traces to learn legitimate agent behaviors and input patterns. From this phase, it derives adaptive policies that regulate tool calls made by the agent, guided by both real-time input context and the control flow dependencies of multi-step agent actions. Evaluation across two real-world AI agent applications demonstrates that AgentGuardian effectively detects malicious or misleading inputs while preserving normal agent functionality. Moreover, its control-flow-based governance mechanism mitigates hallucination-driven errors and other orchestration-level malfunctions.

Figures (5)

• Figure 1: AgentGuardian's conceptual architecture. Orange lines denote the policy generation flow; blue lines denote the enforcement flow.
• Figure 2: Access control policy generation pipeline.
• Figure 3: Example of an access control policy for Senior Data Researcher agent's Read File tool. The time window enforces normal working hours (07:33–20:25). The regex patterns limit access to .txt files within the Cars and AI folders, with the AI folder further restricted to files starting with ai and ending with -2025.
• Figure 4: Example of a CFG for the Senior Data Researcher agent. The agent uses three tools in the following sequence: List Files (first), Read File (second), and Serper Search (third). The allowed paths are listed under the required_leading_contexts node, with each path beginning with a double dash. The Read File and Serper tools can be invoked multiple times, and therefore their repeat field is set to true. Each of these tools has two possible execution paths: one for the initial invocation and another for repeated use, represented by the two branches.
• Figure 5: Regex patterns generated for the File Writer tool’s input parameters. The regex pattern on the left, derived from 10 samples, is overly permissive and effectively accepts any textual input. The regex patterns on the right, generated from 60 samples, is significantly more restrictive. This demonstrates that increasing the number of samples produces a more robust access control policy.