A Methodology for Quantitative AI Risk Modeling

TL;DR

The paper tackles the lack of quantitative AI risk assessment by offering a six-step methodology that merges scenario-building with probabilistic risk estimation. It operationalizes risk as a product of initiation frequency, pathway success probability, and harm, then ties these factors to measurable indicators (KRIs) and LLM uplift through expert elicitation and LLM estimation. A Bayesian-network framework integrates uncertainty and dependencies to produce aggregate risk metrics, enabling concrete claims and informing mitigation and threshold decisions. While demonstrated primarily in AI-enabled cyber offense, the approach aims to generalize to other systemic AI risks and provides a foundation for scalable, evidence-driven risk management in AI deployment.

Abstract

Although general-purpose AI systems offer transformational opportunities in science and industry, they simultaneously raise critical concerns about safety, misuse, and potential loss of control. Despite these risks, methods for assessing and managing them remain underdeveloped. Effective risk management requires systematic modeling to characterize potential harms, as emphasized in frameworks such as the EU General-Purpose AI Code of Practice. This paper advances the risk modeling component of AI risk management by introducing a methodology that integrates scenario building with quantitative risk estimation, drawing on established approaches from other high-risk industries. Our methodology models risks through a six-step process: (1) defining risk scenarios, (2) decomposing them into quantifiable parameters, (3) quantifying baseline risk without AI models, (4) identifying key risk indicators such as benchmarks, (5) mapping these indicators to model parameters to estimate LLM uplift, and (6) aggregating individual parameters into risk estimates that enable concrete claims (e.g., X% probability of >\$Y in annual cyber damages). We examine the choices that underlie our methodology throughout the article, with discussions of strengths, limitations, and implications for future research. Our methodology is designed to be applicable to key systemic AI risks, including cyber offense, biological weapon development, harmful manipulation, and loss-of-control, and is validated through extensive application in LLM-enabled cyber offense. Detailed empirical results and cyber-specific insights are presented in a companion paper.

Figures (3)

• Figure 1: Our risk management methodology first decomposes the risk universe into distinct scenarios, then models each using three types of parameters: the frequency with which a specific sequence of events is initiated, the probability of the sequence taking place, and the harm that would arise as a result.
• Figure 2: An illustrative risk model decomposed into its constituent parameters.
• Figure 3: An illustrative mapping of benchmark performance to risk parameter value, constructed by elicitation with the question “If an LLM could perform all tasks up to task X, what would be the likelihood of event Y, where this LLM is used?” and interpolated.