Designing Incident Reporting Systems for Harms from General-Purpose AI

TL;DR

The paper addresses the governance gap for harms from general-purpose AI by proposing a seven-dimension institutional design framework for AI incident reporting and validating it through nine US safety-critical industry case studies. It systematically analyzes policy goals, actors, incident types, risk materialization, enforcement, anonymity, and post-reporting actions to derive actionable design considerations. The work highlights the importance of near-miss reporting, multi-party data coverage, information sharing standards, and clear liability frameworks to enable safety learning and accountability. By translating cross-industry lessons into GPAI-specific guidance, it aims to inform US researchers and policymakers on how to structure effective incident reporting systems for GPAI harms.

Abstract

We introduce a conceptual framework and provide considerations for the institutional design of AI incident reporting systems, i.e., processes for collecting information about safety- and rights-related events caused by general-purpose AI. As general-purpose AI systems are increasingly adopted, they are causing more real-world harms and displaying the potential to cause significantly more dangerous incidents - events that did or could have caused harm to individuals, property, or the environment. Through a literature review, we develop a framework for understanding the institutional design of AI incident reporting systems, which includes seven dimensions: policy goal, actors submitting and receiving reports, type of incidents reported, level of risk materialization, enforcement of reporting, anonymity of reporters, and post-reporting actions. We then examine nine case studies of incident reporting in safety-critical industries to extract design considerations for AI incident reporting in the United States. We discuss, among other factors, differences in systems operated by regulatory vs. non-regulatory government agencies, near miss reporting, the roles of mandatory reporting thresholds and voluntary reporting channels, how to enable safety learning after reporting, sharing incident information, and clarifying legal frameworks for reporting. Our aim is to inform researchers and policymakers about when particular design choices might be more or less appropriate for AI incident reporting.

Figures (4)

• Figure 1: Lifecycle of an (AI) incident
• Figure 1: Lifecycle of an (AI) incident, i.e., visualization for level of risk materialization, as defined in Table \ref{['tab:Def_Scope_of_Risk']}. Reproduced from Section \ref{['sec:Framework']}.
• Figure 2: Diagram of methodology
• Figure 3: Visualization of incident reporting systems involving different actors, as defined in Table \ref{['tab:Def_Actors']}. Note: each category of reporting system in Table \ref{['tab:Def_Actors']} is represented in the figure by a different color and dashed line (as labelled). Dashed lines are for accessibility only and do not represent additional distinctions beyond the colors and labels. US state and local governments are repeated to emphasize that there are many such governments, all of which may operate independent reporting systems in the same reporting regime in parallel (in contrast, with the notable exception of cybersecurity, systems in most incident reporting regimes are not normally run by multiple different agencies at the federal/international levels).