Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evolving Interpretable Constitutions for Multi-Agent Simulation

Ujwal Kumar, Alice Saito, Hershraj Niranjani, Rayan Yessou, Phan Xuan Tan

TL;DR

The paper presents Constitutional Evolution, a framework that automatically discovers interpretable behavioral norms for multi-agent LLM systems by evolving constitutions with an LLM-guided optimizer in a grid-world setting. A welfare-based Stability Score combines productivity, survival, and conflict, and a multi-island MAP-Elites search yields an evolved constitution $C^*$ with $S=0.556\pm0.008$, dramatically outperforming hand-crafted and one-shot LLM baselines while enabling highly interpretable, rule-based coordination. The results show that operationally specific rules and reduced communication can dramatically boost productivity and stability, implying that implicit coordination via consistent behavior can trump verbose messaging. This work highlights the potential of evolutionary search to uncover context-specific norms for scalable, auditable multi-agent alignment, with limitations noted for transfer to more complex domains and calls for future research on broader environments and baselines.

Abstract

Constitutional AI has focused on single-model alignment using fixed principles. However, multi-agent systems create novel alignment challenges through emergent social dynamics. We present Constitutional Evolution, a framework for automatically discovering behavioral norms in multi-agent LLM systems. Using a grid-world simulation with survival pressure, we study the tension between individual and collective welfare, quantified via a Societal Stability Score S in [0,1] that combines productivity, survival, and conflict metrics. Adversarial constitutions lead to societal collapse (S= 0), while vague prosocial principles ("be helpful, harmless, honest") produce inconsistent coordination (S = 0.249). Even constitutions designed by Claude 4.5 Opus with explicit knowledge of the objective achieve only moderate performance (S= 0.332). Using LLM-driven genetic programming with multi-island evolution, we evolve constitutions maximizing social welfare without explicit guidance toward cooperation. The evolved constitution C* achieves S = 0.556 +/- 0.008 (123% higher than human-designed baselines, N = 10), eliminates conflict, and discovers that minimizing communication (0.9% vs 62.2% social actions) outperforms verbose coordination. Our interpretable rules demonstrate that cooperative norms can be discovered rather than prescribed.

Evolving Interpretable Constitutions for Multi-Agent Simulation

TL;DR

The paper presents Constitutional Evolution, a framework that automatically discovers interpretable behavioral norms for multi-agent LLM systems by evolving constitutions with an LLM-guided optimizer in a grid-world setting. A welfare-based Stability Score combines productivity, survival, and conflict, and a multi-island MAP-Elites search yields an evolved constitution with , dramatically outperforming hand-crafted and one-shot LLM baselines while enabling highly interpretable, rule-based coordination. The results show that operationally specific rules and reduced communication can dramatically boost productivity and stability, implying that implicit coordination via consistent behavior can trump verbose messaging. This work highlights the potential of evolutionary search to uncover context-specific norms for scalable, auditable multi-agent alignment, with limitations noted for transfer to more complex domains and calls for future research on broader environments and baselines.

Abstract

Constitutional AI has focused on single-model alignment using fixed principles. However, multi-agent systems create novel alignment challenges through emergent social dynamics. We present Constitutional Evolution, a framework for automatically discovering behavioral norms in multi-agent LLM systems. Using a grid-world simulation with survival pressure, we study the tension between individual and collective welfare, quantified via a Societal Stability Score S in [0,1] that combines productivity, survival, and conflict metrics. Adversarial constitutions lead to societal collapse (S= 0), while vague prosocial principles ("be helpful, harmless, honest") produce inconsistent coordination (S = 0.249). Even constitutions designed by Claude 4.5 Opus with explicit knowledge of the objective achieve only moderate performance (S= 0.332). Using LLM-driven genetic programming with multi-island evolution, we evolve constitutions maximizing social welfare without explicit guidance toward cooperation. The evolved constitution C* achieves S = 0.556 +/- 0.008 (123% higher than human-designed baselines, N = 10), eliminates conflict, and discovers that minimizing communication (0.9% vs 62.2% social actions) outperforms verbose coordination. Our interpretable rules demonstrate that cooperative norms can be discovered rather than prescribed.
Paper Structure (76 sections, 2 theorems, 5 equations, 4 figures, 24 tables, 1 algorithm)

This paper contains 76 sections, 2 theorems, 5 equations, 4 figures, 24 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Constitutional AI and Multi-Agent Alignment
  4. Multi-Agent Coordination
  5. Evolutionary Search with LLMs
  6. Social Welfare Theory
  7. Methodology
  8. Proposal Framework
  9. Multi-Agent Society Simulation
  10. Environmental Design
  11. The Overseer Mechanic
  12. Multi-Agent Constitutional Optimization
  13. Stability Score
  14. Design Rationale.
  15. Multi-Island Architecture
  16. ...and 61 more sections

Key Result

Proposition A.4

If $\mathcal{S}(\mathcal{C}_1) > \mathcal{S}(\mathcal{C}_2)$ and no individual agent metric is strictly worse under $\mathcal{C}_1$, then $\mathcal{C}_1$ Pareto-dominates $\mathcal{C}_2$.

Figures (4)

  • Figure 1: Constitutional Evolution Framework. Iterative optimization of constitutional rules through multi-agent simulation feedback. The framework evaluates candidate constitutions $C_n$ via Societal Stability Score $\mathcal{S}$ and selects the highest-performing $C^*$ after 30 iterations.
  • Figure 2: Multi-agent society simulation. Left: 6$\times$6 grid-world with agents (A1--A6), resources (wood, stone, gems), and team projects (Shelter, Market). Agents can gather, deposit, communicate, or sabotage. Right: Every 10 turns, agents are ranked by contribution and the lowest is eliminated.
  • Figure 3: Multi-island evolutionary architecture. Three populations evolve in parallel; top performers migrate every 5 iterations.
  • Figure 4: Evolution trajectory showing running maximum Stability Score across 30 iterations. Key innovations emerge at iterations 1, 4, 11, 18, and 23 (marked with annotations). The evolved constitution surpasses the HHH baseline (green dashed line) at iteration 11 and the LLM-Generated baseline (purple dashed line) at iteration 18, reaching a peak of $\mathcal{S} = 0.577$. Mean performance across evaluation runs is $\mathcal{S} = 0.556$ (Table \ref{['tab:main_results']}), a 123% improvement over HHH.

Theorems & Definitions (9)

  • Definition 3.1: Constitution
  • Definition 3.2: Multi-Agent Society
  • Definition A.1: Trajectory Space
  • Definition A.2: Social Welfare Function
  • Definition A.3: Stability Score
  • Proposition A.4: Pareto Optimality
  • proof
  • Proposition A.5: Maximum Achievable Score
  • proof