Benevolent Dictators? On LLM Agent Behavior in Dictator Games
Andreas Einwiller, Kanishka Ghosh Dastidar, Artur Romazanov, Annette Hautli-Janisz, Michael Granitzer, Florian Lemmerich
TL;DR
This paper addresses prompt-induced variability in LLM agent behavior within dictator-game paradigms by introducing the LLM-ABS framework, which systematically varies system prompts, endowment amounts, and units while using neutral user prompts. The method combines open-ended model responses with linguistic analysis (epistemic and discourse markers) and reduces them to structured JSON for robust, cross-model comparisons. Key findings show that system prompts can significantly shift generosity for several models, though a general tendency toward a fair $50/50$ split is observed, with some agents (e.g., Grok, Gemini, Llama) displaying strong altruism and others (e.g., Qwen) showing more self-interest. The work provides a practical baseline for robust LLM agent-behavior studies and highlights the importance of prompt design and linguistic interpretation in deploying AI agents in decision-making tasks.
Abstract
In behavioral sciences, experiments such as the ultimatum game are conducted to assess preferences for fairness or self-interest of study participants. In the dictator game, a simplified version of the ultimatum game where only one of two players makes a single decision, the dictator unilaterally decides how to split a fixed sum of money between themselves and the other player. Although recent studies have explored behavioral patterns of AI agents based on Large Language Models (LLMs) instructed to adopt different personas, we question the robustness of these results. In particular, many of these studies overlook the role of the system prompt - the underlying instructions that shape the model's behavior - and do not account for how sensitive results can be to slight changes in prompts. However, a robust baseline is essential when studying highly complex behavioral aspects of LLMs. To overcome previous limitations, we propose the LLM agent behavior study (LLM-ABS) framework to (i) explore how different system prompts influence model behavior, (ii) get more reliable insights into agent preferences by using neutral prompt variations, and (iii) analyze linguistic features in responses to open-ended instructions by LLM agents to better understand the reasoning behind their behavior. We found that agents often exhibit a strong preference for fairness, as well as a significant impact of the system prompt on their behavior. From a linguistic perspective, we identify that models express their responses differently. Although prompt sensitivity remains a persistent challenge, our proposed framework demonstrates a robust foundation for LLM agent behavior studies. Our code artifacts are available at https://github.com/andreaseinwiller/LLM-ABS.