Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Open Problems in Technical AI Governance

Anka Reuel, Ben Bucknall, Stephen Casper, Tim Fist, Lisa Soder, Onni Aarne, Lewis Hammond, Lujain Ibrahim, Alan Chan, Peter Wills, Markus Anderljung, Ben Garfinkel, Lennart Heim, Andrew Trask, Gabriel Mukobi, Rylan Schaeffer, Mauricio Baker, Sara Hooker, Irene Solaiman, Alexandra Sasha Luccioni, Nitarshan Rajkumar, Nicolas Moës, Jeffrey Ladish, David Bau, Paul Bricman, Neel Guha, Jessica Newman, Yoshua Bengio, Tobin South, Alex Pentland, Sanmi Koyejo, Mykel J. Kochenderfer, Robert Trager

TL;DR

Open Problems in Technical AI Governance defines TAIG, a field dedicated to technical analyses and tools that support AI governance. It presents a two-dimensional taxonomy of capacities (assessment, access, verification, security, operationalization, ecosystem monitoring) and governance targets (data, compute, models, deployment), and enumerates concrete open problems across these axes. The paper argues for a targeted, cross-disciplinary research program to identify intervention points, inform policy choices, and enable enforceable governance mechanisms while warning about dual-use risks and the limits of purely technical fixes. It emphasizes infrastructure for robust evaluation, data provenance, verifiable audits, hardware-enabled governance, and ecosystem monitoring as crucial to improving governance of increasingly capable AI systems. Overall, it provides a structured resource for researchers and funders to direct technical efforts toward governance-relevant challenges.

Abstract

AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. In this paper, we explain what technical AI governance is, why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.

Open Problems in Technical AI Governance

TL;DR

Open Problems in Technical AI Governance defines TAIG, a field dedicated to technical analyses and tools that support AI governance. It presents a two-dimensional taxonomy of capacities (assessment, access, verification, security, operationalization, ecosystem monitoring) and governance targets (data, compute, models, deployment), and enumerates concrete open problems across these axes. The paper argues for a targeted, cross-disciplinary research program to identify intervention points, inform policy choices, and enable enforceable governance mechanisms while warning about dual-use risks and the limits of purely technical fixes. It emphasizes infrastructure for robust evaluation, data provenance, verifiable audits, hardware-enabled governance, and ecosystem monitoring as crucial to improving governance of increasingly capable AI systems. Overall, it provides a structured resource for researchers and funders to direct technical efforts toward governance-relevant challenges.

Abstract

AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. In this paper, we explain what technical AI governance is, why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.
Paper Structure (68 sections, 8 figures, 3 tables)

This paper contains 68 sections, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Relation to AI Governance
  3. Scope and Limitations
  4. Reader Guide
  5. Taxonomy
  6. Assessment
  7. Data
  8. Identification of Problematic Data
  9. Infrastructure and Metadata to Analyze Large Datasets
  10. Attribution of Model Behavior to Data
  11. Compute
  12. Definition of Chip and Cluster Specifications for Model Training
  13. Classification of Workloads
  14. Models and Algorithms
  15. Reliable Evaluations
  16. ...and 53 more sections

Figures (8)

  • Figure 1: An overview of the open problem areas covered in this report, organized according to our taxonomy.
  • Figure 2: Open problem areas in the Assessment capacity, organized by target
  • Figure 3: Open problem areas in the Access capacity, organized by target
  • Figure 4: Open problem areas in the Verification capacity, organized by target
  • Figure 5: Open problem areas in the Security capacity, organized by target
  • ...and 3 more figures