Humans expect rationality and cooperation from LLM opponents in strategic games

TL;DR

This work investigates how humans behave in strategic settings when facing LLM opponents in a multi-player p-beauty contest with no dominant strategy. Using a within-subject design and incentives, the study contrasts human opponents with two LLMs (Chat-GPT v3.5 and Claude v2) and uncovers that subjects, especially those with high strategic reasoning, choose significantly lower numbers against LLMs, driven largely by increased zero-Nash-equilibrium choices. Qualitative and prediction-data reveal that perceived LLM sophistication leads participants to expect rational and sometimes cooperative behavior from LLMs, shaping beliefs and actions. These findings have important implications for mechanism design in mixed human-LLM environments and motivate further research across a broader set of strategic contexts and models. Key results include: (i) identification of a subset (16.7%) of participants with high strategic reasoning ability using three criteria, (ii) a substantial shift toward lower play against LLMs (mean difference of ~$6$ points, with $15.3\%$ zeros against LLMs vs $4.2\%$ against humans), and (iii) clearer motivation patterns where reasoning and cooperation beliefs about LLMs emerge as drivers of behavior. The work highlights the need to account for heterogeneity in beliefs about LLMs when designing economic mechanisms in mixed human-LLM ecosystems, and sets the stage for broader exploration of strategic human-LLM interactions.

Abstract

As Large Language Models (LLMs) integrate into our social and economic interactions, we need to deepen our understanding of how humans respond to LLMs opponents in strategic settings. We present the results of the first controlled monetarily-incentivised laboratory experiment looking at differences in human behaviour in a multi-player p-beauty contest against other humans and LLMs. We use a within-subject design in order to compare behaviour at the individual level. We show that, in this environment, human subjects choose significantly lower numbers when playing against LLMs than humans, which is mainly driven by the increased prevalence of `zero' Nash-equilibrium choices. This shift is mainly driven by subjects with high strategic reasoning ability. Subjects who play the zero Nash-equilibrium choice motivate their strategy by appealing to perceived LLM's reasoning ability and, unexpectedly, propensity towards cooperation. Our findings provide foundational insights into the multi-player human-LLM interaction in simultaneous choice games, uncover heterogeneities in both subjects' behaviour and beliefs about LLM's play when playing against them, and suggest important implications for mechanism design in mixed human-LLM systems.

Figures (5)

• Figure 1: Schematic summary of experimental design.
• Figure 2: G3P2/3 task when playing against humans and LLMs: (a) Frequency histograms of observed choices and (b) subject-specific differences between choices.
• Figure 3: Schema showing the number (and proportion) of subjects who meet the three conditions we impose on individual decisions to identify subjects with high strategic reasoning ability. To be classified as displaying high strategic reasoning ability, a subject must satisfy all three conditions -- A, B, and C.
• Figure 4: Interface of the on-screen calculator used in the experiment.
• Figure 5: Interface of the predictions' elicitation task of the experiment.