Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas

TL;DR

This survey addresses cooperation in social dilemmas across AI and human contexts, formalizing normal-form and sequential models with $R$, $P$, $T$, $S$ payoffs and stochastic environments. It synthesizes three strands: multi-agent cooperation in SSDs through shaping intrinsic/external motivations and opponent adaptation; human-agent cooperation via algorithm design and human biases; and leveraging AI to promote human-human cooperation, including mechanisms such as inequity aversion, altruism, SVO, social influence, reputation, gifting, and contracts. The authors review representative SSD environments (Gathering, Wolfpack, Coins, CleanUp, Harvest) and analyze strategies for promoting cooperation, such as amTFT/LOLA-style opponent shaping and S++/S# signaling, while highlighting empirical insights from human-agent interactions. They conclude with future directions, emphasizing the role of large language models, theoretical grounding, real-world applications (e.g., autonomous driving), and bridging human-agent cooperation with SSDs to advance cooperative AI and policy design.

Abstract

The study of cooperation within social dilemmas has long been a fundamental topic across various disciplines, including computer science and social science. Recent advancements in Artificial Intelligence (AI) have significantly reshaped this field, offering fresh insights into understanding and enhancing cooperation. This survey examines three key areas at the intersection of AI and cooperation in social dilemmas. First, focusing on multi-agent cooperation, we review the intrinsic and external motivations that support cooperation among rational agents, and the methods employed to develop effective strategies against diverse opponents. Second, looking into human-agent cooperation, we discuss the current AI algorithms for cooperating with humans and the human biases towards AI agents. Third, we review the emergent field of leveraging AI agents to enhance cooperation among humans. We conclude by discussing future research avenues, such as using large language models, establishing unified theoretical frameworks, revisiting existing theories of human cooperation, and exploring multiple real-world applications.

Figures (6)

• Figure 1: Understanding cooperation within multi-agent, human-agent systems, and beyond. (A) Normal-form social dilemmas and sequential social dilemmas. (B) Two approaches to solving sequential social dilemmas in multi-agent systems: i) promoting cooperation among agents through shaping their intrinsic and external motivations; ii) devising and selecting strategies in response to diverse opponents. (C) Four perspectives of studying cooperation in human-agent hybrid systems: i) designing algorithms for cooperating with humans; ii) identifying and mitigating human biases in human-agent cooperation; iii) scaffolding cooperation in human-human interactions, e.g., by engineering the interaction structure; iv) delegating human decision making to agents.
• Figure 2: Illustration of Gathering leibo2017multi.
• Figure 3: Illustration of Wolfpack leibo2017multi.
• Figure 4: Illustration of Coins lerer2017maintaining.
• Figure 5: Illustration of CleanUp hughes2018inequity.