AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Making

TL;DR

AdaSociety presents an adaptive multi-agent platform that jointly expands physical state/action spaces and adapts social connections, enabling massive task diversity driven by agent behavior. It formalizes Growing-MG on top of a multilayer social-graph framework to capture dynamic coalitions, hierarchy, and information/reward flows, and offers three built-in mini-games to probe social structure, contract formation, and negotiation. Early experiments with PPO, RecPPO, MAPPO, curriculum learning, and an LLM-based controller reveal that social structures can yield functional benefits, though current RL and LLM methods often struggle to effectively exploit social dynamics, underscoring the need for further algorithmic development. AdaSociety serves as a versatile research platform for studying co-evolution, coalition formation, and social intelligence in heterogeneous agent systems, with open-source code to enable broader experimentation and extension.

Abstract

Traditional interactive environments limit agents' intelligence growth with fixed tasks. Recently, single-agent environments address this by generating new tasks based on agent actions, enhancing task diversity. We consider the decision-making problem in multi-agent settings, where tasks are further influenced by social connections, affecting rewards and information access. However, existing multi-agent environments lack a combination of adaptive physical surroundings and social connections, hindering the learning of intelligent behaviors. To address this, we introduce AdaSociety, a customizable multi-agent environment featuring expanding state and action spaces, alongside explicit and alterable social structures. As agents progress, the environment adaptively generates new tasks with social structures for agents to undertake. In AdaSociety, we develop three mini-games showcasing distinct social structures and tasks. Initial results demonstrate that specific social structures can promote both individual and collective benefits, though current reinforcement learning and LLM-based algorithms show limited effectiveness in leveraging social structures to enhance performance. Overall, AdaSociety serves as a valuable research platform for exploring intelligence in diverse physical and social settings. The code is available at https://github.com/bigai-ai/AdaSociety.

Figures (14)

• Figure 1: An overview of AdaSociety, composed of physical component and social component. Physical Component consists of diverse resources and events on the map and heterogeneous agents' inventories. Social Component describes the adaptive connections between agents and organizations, which shape information access and reward structure. Agents take social actions to alter their social connections. As shown in the rightmost flowchart, agents are initially independent and can establish individual connections (edges between nodes) and form groups (gray ovals).
• Figure 2: Overview of three mini-games.
• Figure 3: Dynamic structure: (a) Individual reward per step with different social structures using 100 samples from PPO-trained policies, (b) Individual reward per step using 100 samples from different policies (c) Learning curves using different learning methods.
• Figure 4: Illustration of a synthesis tree.
• Figure 5: Illustration of the mutual adaptation between agents and AdaSociety.