Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Computational Compliance for AI Regulation: Blueprint for a New Research Domain

Bill Marino, Nicholas D. Lane

TL;DR

The paper argues that scalable AIR compliance requires computational methods that operate across an AI system’s life cycle. It introduces CAIRC, a closed-loop framework with an Inspector (air-diagnostic) and a Mechanic (repair engine) that iteratively restore compliance. It defines design criteria for inputs/outputs and mapping functions, and proposes a benchmark dataset of AI system snapshots to quantify progress along multiple dimensions, including deployability and end-to-end loop effectiveness. The work highlights limitations and outlines concrete future directions to advance a nascent field aimed at real-time, regulatorily aligned AI systems with broad practical impact.

Abstract

The era of AI regulation (AIR) is upon us. But AI systems, we argue, will not be able to comply with these regulations at the necessary speed and scale by continuing to rely on traditional, analogue methods of compliance. Instead, we posit that compliance with these regulations will only realistically be achieved computationally: that is, with algorithms that run across the life cycle of an AI system, automatically steering it toward AIR compliance in the face of dynamic conditions. Yet despite their (we would argue) inevitability, the research community has yet to specify exactly how these algorithms for computational AIR compliance should behave - or how we should benchmark their performance. To fill these gaps, we specify a set of design goals for such algorithms. In addition, we specify a benchmark dataset that can be used to quantitatively measure whether individual algorithms satisfy these design goals. By delivering this blueprint, we hope to give shape to an important but uncrystallized new domain of research - and, in doing so, incite necessary investment in it.

Computational Compliance for AI Regulation: Blueprint for a New Research Domain

TL;DR

The paper argues that scalable AIR compliance requires computational methods that operate across an AI system’s life cycle. It introduces CAIRC, a closed-loop framework with an Inspector (air-diagnostic) and a Mechanic (repair engine) that iteratively restore compliance. It defines design criteria for inputs/outputs and mapping functions, and proposes a benchmark dataset of AI system snapshots to quantify progress along multiple dimensions, including deployability and end-to-end loop effectiveness. The work highlights limitations and outlines concrete future directions to advance a nascent field aimed at real-time, regulatorily aligned AI systems with broad practical impact.

Abstract

The era of AI regulation (AIR) is upon us. But AI systems, we argue, will not be able to comply with these regulations at the necessary speed and scale by continuing to rely on traditional, analogue methods of compliance. Instead, we posit that compliance with these regulations will only realistically be achieved computationally: that is, with algorithms that run across the life cycle of an AI system, automatically steering it toward AIR compliance in the face of dynamic conditions. Yet despite their (we would argue) inevitability, the research community has yet to specify exactly how these algorithms for computational AIR compliance should behave - or how we should benchmark their performance. To fill these gaps, we specify a set of design goals for such algorithms. In addition, we specify a benchmark dataset that can be used to quantitatively measure whether individual algorithms satisfy these design goals. By delivering this blueprint, we hope to give shape to an important but uncrystallized new domain of research - and, in doing so, incite necessary investment in it.
Paper Structure (38 sections, 1 figure)

This paper contains 38 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Why Computational AIR Compliance Is Inevitable
  3. Deconstructing the problem
  4. The Inspector
  5. Input
  6. Comprehensive
  7. Concurrent
  8. Attestable
  9. Output
  10. Mechanic-enabling
  11. Function mapping input to output
  12. High accuracy, precision, and recall
  13. Low latency
  14. The Mechanic
  15. Input
  16. ...and 23 more sections

Figures (1)

  • Figure 1: CAIRC flowchart. As a first step, information about an AI system is submitted (e.g., by an overarching CAIRC algorithm) to an Inspector. Next, the Inspector reaches a finding of either compliance, in which case the process is complete (for the time being), or non-compliance, in which case the Inspector transmits its diagnosis to the Mechanic. Upon receiving this diagnosis, the Mechanic uses one or more automated tools to try to repair the diagnosed compliance defect(s). When finished, it calls the Inspector to re-run its analysis. This loop repeats until the Inspector finds that compliance exists, in which case the process has concluded (until, at least, it is triggered again).