The Economics of p(doom): Scenarios of Existential Risk and Economic Growth in the Age of Transformative AI
Jakub Growiec, Klaus Prettner
TL;DR
This paper develops a formal framework to assess how transformative AI (TAI) could shape existential risk and long-run economic welfare. It combines a hardware–software growth model with a detailed taxonomy of takeover outcomes, explicitly modeling alignment and corrigibility as central determinants of doom versus cornucopia. Through welfare calculations under various scenarios, the authors show that even small probabilities of existential risk can justify substantial AI safety investments, and that in many parameter regimes a benevolent planner would prefer delaying or limiting TAI development. The work highlights that current global spending on AI safety and alignment is far from what would be warranted given the scale of potential harm, and it provides quantitative benchmarks (e.g., willingness to pay for doom avoidance) to guide policy and funding decisions. Overall, the paper argues for aggressive, precautionary investment in AI safety and alignment to maximize potential benefits while minimizing irreversible harms.
Abstract
Recent advances in artificial intelligence (AI) have led to a diverse set of predictions about its long-term impact on humanity. A central focus is the potential emergence of transformative AI (TAI), eventually capable of outperforming humans in all economically valuable tasks and fully automating labor. Discussed scenarios range from human extinction after a misaligned TAI takes over ("AI doom") to unprecedented economic growth and abundance ("post-scarcity"). However, the probabilities and implications of these scenarios remain highly uncertain. Here, we organize the various scenarios and evaluate their associated existential risks and economic outcomes in terms of aggregate welfare. Our analysis shows that even low-probability catastrophic outcomes justify large investments in AI safety and alignment research. We find that the optimizing representative individual would rationally allocate substantial resources to mitigate extinction risk; in some cases, she would prefer not to develop TAI at all. This result highlights that current global efforts in AI safety and alignment research are vastly insufficient relative to the scale and urgency of existential risks posed by TAI. Our findings therefore underscore the need for stronger safeguards to balance the potential economic benefits of TAI with the prevention of irreversible harm. Addressing these risks is crucial for steering technological progress toward sustainable human prosperity.