Quantitative AI Risk Assessments: Opportunities and Challenges

TL;DR

The paper addresses the need for quantitative AI risk assessments amid rising regulatory and societal concerns around AI. It argues for integrating 'as-is' quantitative risk measurements with existing governance to evaluate high-stakes AI systems when full development data is unavailable, by defining risk dimensions such as Performance, Fairness, Privacy, Adversarial Robustness, Explainability, and Value Alignment. It provides design principles for metrics (distinguishing individual vs. summary metrics) and outlines practical considerations for metric selection, interpretation, thresholding, and reporting, including regulatory interplay and implementation challenges. The work aims to enable objective, consistent, and regulator-relevant evaluations that can be embedded into existing risk processes, while acknowledging tradeoffs, information loss, and the need for context-aware, user-centered summaries.

Abstract

Although AI systems are increasingly being leveraged to provide value to organizations, individuals, and society, significant attendant risks have been identified and have manifested. These risks have led to proposed regulations, litigation, and general societal concerns. As with any promising technology, organizations want to benefit from the positive capabilities of AI technology while reducing the risks. The best way to reduce risks is to implement comprehensive AI lifecycle governance where policies and procedures are described and enforced during the design, development, deployment, and monitoring of an AI system. Although support for comprehensive governance is beginning to emerge, organizations often need to identify the risks of deploying an already-built model without knowledge of how it was constructed or access to its original developers. Such an assessment will quantitatively assess the risks of an existing model in a manner analogous to how a home inspector might assess the risks of an already-built home or a physician might assess overall patient health based on a battery of tests. Several AI risks can be quantified using metrics from the technical community. However, there are numerous issues in deciding how these metrics can be leveraged to create a quantitative AI risk assessment. This paper explores these issues, focusing on the opportunities, challenges, and potential impacts of such an approach, and discussing how it might influence AI regulations.