With Great Capabilities Come Great Responsibilities: Introducing the Agentic Risk & Capability Framework for Governing Agentic AI Systems
Shaun Khoo, Jessica Foo, Roy Ka-Wei Lee
TL;DR
The paper tackles the challenge of governing agentic AI systems that act autonomously by introducing the Agentic Risk & Capability (ARC) framework, which uses a capability-centric lens to connect system elements (components, design, capabilities) with risks (failure modes, hazards) and technical controls. It provides a structured Risk Register, tiered technical controls, and an implementation pathway, complemented by worked examples (Researcher and Vibe Coder) to illustrate practical risk identification and mitigation. ARC aims to enable rapid, scalable, and responsible deployment of agentic AI by offering context-aware risk assessment, actionable controls, and a framework adaptable to evolving capabilities. The work emphasizes practitioner-oriented artifacts, forward compatibility with governance needs, and future directions for empirical validation and automation of risk management.
Abstract
Agentic AI systems present both significant opportunities and novel risks due to their capacity for autonomous action, encompassing tasks such as code execution, internet interaction, and file modification. This poses considerable challenges for effective organizational governance, particularly in comprehensively identifying, assessing, and mitigating diverse and evolving risks. To tackle this, we introduce the Agentic Risk \& Capability (ARC) Framework, a technical governance framework designed to help organizations identify, assess, and mitigate risks arising from agentic AI systems. The framework's core contributions are: (1) it develops a novel capability-centric perspective to analyze a wide range of agentic AI systems; (2) it distills three primary sources of risk intrinsic to agentic AI systems - components, design, and capabilities; (3) it establishes a clear nexus between each risk source, specific materialized risks, and corresponding technical controls; and (4) it provides a structured and practical approach to help organizations implement the framework. This framework provides a robust and adaptable methodology for organizations to navigate the complexities of agentic AI, enabling rapid and effective innovation while ensuring the safe, secure, and responsible deployment of agentic AI systems. Our framework is open-sourced \href{https://govtech-responsibleai.github.io/agentic-risk-capability-framework/}{here}.