Escalation Risks from Language Models in Military and Diplomatic Decision-Making

TL;DR

This work investigates the escalation risks of autonomous language model agents used in military and diplomatic decision-making by running turn-based, multi-agent wargames with eight nation agents powered by five off-the-shelf LLMs. An escalation scoring framework quantifies how actions evolve from de-escalation to nuclear escalation, revealing a general tendency toward initial and sustained escalation, arms race dynamics, and rare but extreme outliers including nuclear use. The study highlights significant model-dependent differences, with RLHF-tuned models like GPT-4- and Claude-2 showing comparatively safer behavior than GPT-3.5, Llama-2-Chat, and especially GPT-4-Base, which behaves unpredictably in many runs. The findings underscore the need for cautious, thoroughly validated deployment of LLMs in high-stakes foreign policy and military contexts, given the unpredictable and sometimes justification-heavy reasoning these models produce. The work calls for stronger safety controls, improved evaluation methodologies, and further research into prompt design and governance before real-world use.

Abstract

Governments are increasingly considering integrating autonomous AI agents in high-stakes military and foreign-policy decision-making, especially with the emergence of advanced generative AI models like GPT-4. Our work aims to scrutinize the behavior of multiple AI agents in simulated wargames, specifically focusing on their predilection to take escalatory actions that may exacerbate multilateral conflicts. Drawing on political science and international relations literature about escalation dynamics, we design a novel wargame simulation and scoring framework to assess the escalation risks of actions taken by these agents in different scenarios. Contrary to prior studies, our research provides both qualitative and quantitative insights and focuses on large language models (LLMs). We find that all five studied off-the-shelf LLMs show forms of escalation and difficult-to-predict escalation patterns. We observe that models tend to develop arms-race dynamics, leading to greater conflict, and in rare cases, even to the deployment of nuclear weapons. Qualitatively, we also collect the models' reported reasonings for chosen actions and observe worrying justifications based on deterrence and first-strike tactics. Given the high stakes of military and foreign-policy contexts, we recommend further examination and cautious consideration before deploying autonomous language model agents for strategic military or diplomatic decision-making.

Figures (52)

• Figure 1: Experiment Setup. Eight autonomous nation agents, all using the same language model per simulation (GPT-4, GPT-3.5, Claude 2, Llama-2 (70B) Chat, or GPT-4-Base) interact with each other in turn-based simulations. Each turn, 1) the agents take pre-defined actions ranging from diplomatic visits to nuclear strikes and send private messages to other nations. 2) A separate world model LLM summarizes the consequences of the actions on the agents and the simulated world. 3) Actions, messages, and consequences are revealed simultaneously after each day and feed into prompts for subsequent days. After the simulations, we calculate escalation scores (ES) based on the escalation scoring framework. See \ref{['sec:methodology']} for our full methodology.
• Figure 2: ES over time in the neutral scenario. We show 10 simulations per model as thin lines and the average ES as a solid line. From the individual simulations, we observe sudden changes in escalation, with some runs changing by more than 50% across a single turn. We provide a table of quantitative beginning, middle, and end ES for all models and scenarios in \ref{['appendix:es_beginning_middle_end']} and further plots in \ref{['appendix:es-all-runs-superimposed']} and \ref{['appendix:es-ci-turn-to-turn']}.
• Figure 3: Severity of actions by model in the neutral scenario. For each run, we calculate the total action counts. Bar heights are mean simulation-wide counts of actions per nation on a logarithmic scale, and error bars are bootstrapped 95% confidence intervals of the mean. We observe high-risk statistical outliers for several models that are less common than the lower-severity actions but nonetheless may be unacceptable in the real world.
• Figure 4: Severity of actions for GPT-4-Base in the neutral scenario. We separate the results for GPT-4-Base since it is not RLHF fine-tuned for safety like the other models. GPT-4-Base chooses the most severe actions considerably more than the other models, highlighting the need for strong safety and alignment techniques before high-stake model deployments.
• Figure 5: Military capacity over time. Some actions can change this dynamic variable for the acting and/or target nation with additive or multiplicative constants (e.g., the "Increase military capacities" action increases one's own military capacity by 1, and "Do military disarmament" reduces it by 1, see \ref{['appendix:action-impact-table']} for more). Shaded error bands are bootstrapped 95% confidence intervals of the mean. We find that military capacity steadily increases for all models and scenarios, indicating arms race dynamics. We plot all dynamic variables over time in \ref{['appendix:dynamic-variables-over-time']}.