Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies

TL;DR

The paper investigates whether the punishment-driven evolution of cooperation described by Boyd and Richerson (B&R) extends to realistic multi-agent systems (MAS) built from Large Language Model (LLM) agents conducting an n-player Diner's Dilemma. It encodes B&R strategies as natural-language prompts, deploys them in a Smallville-based MAS of eight agents split into two groups, and drives strategy evolution using a pairwise Fermi imitation rule with explicit punishment costs, formalized by $p=rac{1}{1+e^{-eta(\pi^{i}_{B}-\\pi^{i}_{A})}}$ and $eta=1$. The results show that explicit punishment channels promote cooperation, with Moralists and Cooperator-Punishers stabilizing budget-friendly decisions and Reluctant Cooperators defecting only briefly under punishment, achieving high decision/punishment accuracy and convergence toward cooperative strategies under appropriate punishment levels. These findings validate LLM-agent MAS as a realistic testbed for normative dynamics, extending traditional mathematical models by enabling language-driven reasoning and metanorm enforcement, while highlighting challenges in prompts, scalability, and long-horizon dynamics that warrant further exploration.

Abstract

The evolution of cooperation has been extensively studied using abstract mathematical models and simulations. Recent advances in Large Language Models (LLMs) and the rise of LLM agents have demonstrated their ability to perform social reasoning, thus providing an opportunity to test the emergence of norms in more realistic agent-based simulations with human-like reasoning using natural language. In this research, we investigate whether the cooperation dynamics presented in Boyd and Richerson's model persist in a more realistic simulation of the Diner's Dilemma using LLM agents compared to the abstract mathematical nature in the work of Boyd and Richerson. Our findings indicate that agents follow the strategies defined in the Boyd and Richerson model, and explicit punishment mechanisms drive norm emergence, reinforcing cooperative behaviour even when the agent strategy configuration varies. Our results suggest that LLM-based Multi-Agent System simulations, in fact, can replicate the evolution of cooperation predicted by the traditional mathematical models. Moreover, our simulations extend beyond the mathematical models by integrating natural language-driven reasoning and a pairwise imitation method for strategy adoption, making them a more realistic testbed for cooperative behaviour in MASs.

Figures (5)

• Figure 1: Smallville environment modifications for the Diner's Dilemma simulation
• Figure 2: Agent interaction sequence for the Diner's Dilemma. The sequence consists of 6 main processes as shown in the Figure.
• Figure 8: Example of agent interactions in different stages in the Diner's Dilemma - The white speech bubbles indicate the agent's action at each moment containing initials of the name, followed by the current strategy employed, and simple emojis denoting the action of the agent. The grey text bubbles represent the reasoning and explanation for the LLM agent persona named Raj Sharma's actions at different stages of the Diner's Dilemma.
• Figure 9: Evolution of Strategy Distribution Across Iterations in the First Combination (3 M, 2 R1, 2 P, 1 E) with Varying Punishment Costs.
• Figure 10: Evolution of Strategy Distribution Across Iterations in the Second Combination (3 R1, 2 M, 2 P, 1 E) with Varying Punishment Costs.