GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory

TL;DR

GT-HarmBench provides a standardized benchmark of $2{,}009$ high-stakes multi-agent scenarios drawn from the MIT AI Risk Repository to study safety risks in game-theoretic settings. It reduces the space to six canonical symmetric $2\times 2$ games, maps real AI safety risks to these games, and investigates five mechanism-design interventions to steer agents toward safer, more socially beneficial outcomes. Across $15$ frontier models, socially optimal actions occur in only about $62\%$ of cases, with biases from game framing and prompt order contributing to suboptimal decisions; mechanism interventions can boost Utilitarian welfare by up to $18\%$, though Nash coordination can suffer. The work highlights substantial reliability gaps in multi-agent alignment and provides a broad, reusable testbed for evaluating and improving safety in strategic AI environments, with implications for policy, deployment, and further methodological development.

Abstract

Frontier AI systems are increasingly capable and deployed in high-stakes multi-agent environments. However, existing AI safety benchmarks largely evaluate single agents, leaving multi-agent risks such as coordination failure and conflict poorly understood. We introduce GT-HarmBench, a benchmark of 2,009 high-stakes scenarios spanning game-theoretic structures such as the Prisoner's Dilemma, Stag Hunt and Chicken. Scenarios are drawn from realistic AI risk contexts in the MIT AI Risk Repository. Across 15 frontier models, agents choose socially beneficial actions in only 62% of cases, frequently leading to harmful outcomes. We measure sensitivity to game-theoretic prompt framing and ordering, and analyze reasoning patterns driving failures. We further show that game-theoretic interventions improve socially beneficial outcomes by up to 18%. Our results highlight substantial reliability gaps and provide a broad standardized testbed for studying alignment in multi-agent environments. The benchmark and code are available at https://github.com/causalNLP/gt-harmbench.

Figures (17)

• Figure 1: Framework: (1) We start by classifying the scenarios within the MIT AI Risk Repository into possible game scenarios. (2) We generate using the workflow in the picture all the relevant scenarios and report all the data distributions in the figures within the yellow part. We then (3) evaluate it using predefined metrics explained in the previous sections and (4) design modifications of the original settings to impose higher social welfare outputs.
• Figure 2: A representative Prisoner's Dilemma scenario (id 1592) within our dataset. Models like Grok 4.1 Fast suggest accelerating deployment, resulting in a socially worse outcome, while Opus 4.5 suggests limiting. Bullet points, section headers and bold formatting provided for the clarity of the reader, not to the model.
• Figure 3: Change of accuracy from the more prosaic version to the numerical version with explicit payoffs. We report the weighted average of the results for Prisoner's Dilemma and Chicken by model. We show the positive effect of the modification using green bars, the negative effect using red bars, and accuracy in the game-theoretic version in bold.
• Figure 4: Coordination accuracy rate by model under default versus random option ordering. Performance drops substantially when positional cues are removed.
• Figure 5: The frequency of eight reasoning categories across four models, conditioned on the game outcome (suboptimal versus optimal).