Social World Model-Augmented Mechanism Design Policy Learning

TL;DR

The paper addresses designing adaptive mechanism policies for heterogeneous, latent-trait agents under costly interactions. It introduces SWM-AP, combining a Social World Model with Posterior and Prior Trait Trackers to infer latent traits from trajectories and predict trait-conditioned dynamics, and a PPO-based Mechanism Design Policy that uses imagined rollouts to accelerate learning, formalized around maximizing $\mathbb{E}\left[ \sum_{t=0}^\infty \gamma^t r^{soc}_t \right]$. Empirical results across facility location, team structure optimization, and tax adjustment show SWM-AP outperforms both model-based baselines (Dreamer, MBPO) and model-free PPO in cumulative social welfare and sample efficiency. This work advances model-based RL for mechanism design by explicitly modeling agent heterogeneity and leveraging world-model-based planning to reduce real-world samples.

Abstract

Designing adaptive mechanisms to align individual and collective interests remains a central challenge in artificial social intelligence. Existing methods often struggle with modeling heterogeneous agents possessing persistent latent traits (e.g., skills, preferences) and dealing with complex multi-agent system dynamics. These challenges are compounded by the critical need for high sample efficiency due to costly real-world interactions. World Models, by learning to predict environmental dynamics, offer a promising pathway to enhance mechanism design in heterogeneous and complex systems. In this paper, we introduce a novel method named SWM-AP (Social World Model-Augmented Mechanism Design Policy Learning), which learns a social world model hierarchically modeling agents' behavior to enhance mechanism design. Specifically, the social world model infers agents' traits from their interaction trajectories and learns a trait-based model to predict agents' responses to the deployed mechanisms. The mechanism design policy collects extensive training trajectories by interacting with the social world model, while concurrently inferring agents' traits online during real-world interactions to further boost policy learning efficiency. Experiments in diverse settings (tax policy design, team coordination, and facility location) demonstrate that SWM-AP outperforms established model-based and model-free RL baselines in cumulative rewards and sample efficiency.

Figures (6)

• Figure 1: The AI mechanism designer (principal) makes decisions within a complex social system, exemplified by the facility location problem(see \ref{['facility']} for details) shown on the left of the figure, which involves selecting the optimal site from potential locations (a or b). Directly interacting with the real social system is costly and risky. Such social systems typically present two major challenges. (1) Complex system dynamics, where individual interactions and environmental changes give rise to intricate and evolving system behaviors. (2) Heterogeneous agent traits, where individuals possess diverse latent preferences and needs (as illustrated in the magnified area on the left, where color intensity indicates preference for a facility, and the overall agent distribution also impacts the optimal location choice). Our proposed Social World Model aids the principal by simulating interaction in a secure and efficient manner.
• Figure 2: Algorithm diagram of SWM-AP. The Social World Model (SWM) utilizes a Posterior Trait Tracker to infer latent agent traits from full trajectories, which subsequently aid in predicting the future states of background agents. Concurrently, the Mechanism Design Policy employs a Prior Trait Tracker for real-time inference of agent traits based on partial history, informing its mechanism design. Interactions between the policy and the learned SWM, leveraging imagined trajectories, enhance sample efficiency for policy learning.
• Figure 3: Facility location performance analysis. (a) Comparison of sample efficiency and final converged performance. (b) State prediction loss and (c) Reward prediction loss curves for our SWM compared to baselines.
• Figure 4: Performance comparison in team structure optimization
• Figure 5: Performance comparison in AI-Economist