Why and How Governments Should Monitor AI Development

TL;DR

The paper argues that government governance of AI is hampered by a lack of timely, high-quality information. It proposes building in-house measurement and monitoring infrastructure to track deployed AI capabilities, research progress, and potential societal impacts, with pilot projects to validate approaches. By providing data-driven insights, the approach seeks to accelerate safe, beneficial AI deployment, enable regulatory conformance, and offer early warnings of risks or opportunities. The authors emphasize a hybrid model—government leadership with collaboration from third parties—to iteratively implement and refine the infrastructure within policymaking processes.

Abstract

In this paper we outline a proposal for improving the governance of artificial intelligence (AI) by investing in government capacity to systematically measure and monitor the capabilities and impacts of AI systems. If adopted, this would give governments greater information about the AI ecosystem, equipping them to more effectively direct AI development and deployment in the most societally and economically beneficial directions. It would also create infrastructure that could rapidly identify potential threats or harms that could occur as a consequence of changes in the AI ecosystem, such as the emergence of strategically transformative capabilities, or the deployment of harmful systems. We begin by outlining the problem which motivates this proposal: in brief, traditional governance approaches struggle to keep pace with the speed of progress in AI. We then present our proposal for addressing this problem: governments must invest in measurement and monitoring infrastructure. We discuss this proposal in detail, outlining what specific things governments could focus on measuring and monitoring, and the kinds of benefits this would generate for policymaking. Finally, we outline some potential pilot projects and some considerations for implementing this in practice.