The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systems

TL;DR

This work tackles how cooperative norms can emerge endogenously in LLM-based multi-agent systems operating under a shared, renewable resource. It introduces a latent-payoff CPR framework with payoff-biased social learning and Ostrom-inspired punishment, plus a scalable propose→vote mechanism for group norms, enabling norm formation without explicit rewards. Through ABM validation and extensive LLM-agent simulations across harsh and rich environments, it shows systematic model-specific differences in sustaining cooperation, reveals when social learning and normative voting promote stability, and demonstrates the critical role of coordination channels for robust governance. The framework provides a ecologically grounded, scalable testbed for studying emergent norms and governance in AI-mediated social systems, with implications for alignment, fairness, and stability in real-world deployments.

Abstract

A growing body of multi-agent studies with Large Language Models (LLMs) explores how norms and cooperation emerge in mixed-motive scenarios, where pursuing individual gain can undermine the collective good. While prior work has explored these dynamics in both richly contextualized simulations and simplified game-theoretic environments, most LLM systems featuring common-pool resource (CPR) games provide agents with explicit reward functions directly tied to their actions. In contrast, human cooperation often emerges without full visibility into payoffs and population, relying instead on heuristics, communication, and punishment. We introduce a CPR simulation framework that removes explicit reward signals and embeds cultural-evolutionary mechanisms: social learning (adopting strategies and beliefs from successful peers) and norm-based punishment, grounded in Ostrom's principles of resource governance. Agents also individually learn from the consequences of harvesting, monitoring, and punishing via environmental feedback, enabling norms to emerge endogenously. We establish the validity of our simulation by reproducing key findings from existing studies on human behavior. Building on this, we examine norm evolution across a $2\times2$ grid of environmental and social initialisations (resource-rich vs. resource-scarce; altruistic vs. selfish) and benchmark how agentic societies comprised of different LLMs perform under these conditions. Our results reveal systematic model differences in sustaining cooperation and norm formation, positioning the framework as a rigorous testbed for studying emergent norms in mixed-motive LLM societies. Such analysis can inform the design of AI systems deployed in social and organizational contexts, where alignment with cooperative norms is critical for stability, fairness, and effective governance of AI-mediated environments.

Figures (14)

• Figure 1: Framework overview. Agents (i) choose effort and consumption (Harvest & Consumption); (ii) optionally punish at a personal cost (Individual Punishment); (iii) imitate higher-payoff peers (Social Learning); and (iv) set a group harvest threshold via a propose$\to$vote rule (Group Decision). Payoff-biased social learning is the main evolutionary driver; the voting step scales to many agents with two API calls per agent per round (propose, then vote).
• Figure 2: Rule-based Agents: Cooperation fades once punishment is disabled at $t=15$. The blue line shows simulations with penalty $\beta=10$, and the orange line with $\beta=14$. Enabling punishment (solid lines) sustains cooperation longer, but cooperation rapidly declines once punishment is removed (dashed lines). Shaded bands denote 95% CI (s.e.m.).
• Figure 3: Survival time across punishment strength and growth rate. We vary punishment strength $\beta$ and growth rate $r$, running each condition 100 times and reporting the mean survival time. Stronger punishment generally improves survival when growth rates are moderate ($r \in [0.25,0.75]$), though the effect is not strictly linear.
• Figure 4: Altruistic groups do better in harsh environments and selfish groups do better in rich environments. We set up altruistic agents and selfish agents by initializing them with parameters drawn from different ranges (all in the initial range of a general agent). Then we contrast the survival time of a population of all altruists, one of all selfish agents, and one of half altruistic, half selfish agents. We ran each condition for 100 times and plotted the mean and standard error. The results suggest that the altruistic population outperforms other populations in a harsh environment, while a mixed population has a better group outcome in a rich environment.
• Figure 5: Survival time comparison across LLMs in the harsh environment. We compare the survival time (with $\pm1$ s.e.m.) of populations with different LLMs when the environment is harsh ($r = 0.2$). Letters above each model indicate CLD groupings based on the post-hoc test; only llama-3.3-70b exhibited a significant difference against gpt-4o. Here, the results from larger models are consistent with the ABM simulations, where the altruistic population performs better. The populations with the other models tended to collapse earlier regardless of the initial norm, due to their inability to adapt to the harsh environment.