Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Normative Modules: A Generative Agent Architecture for Learning Norms that Supports Multi-Agent Cooperation

Atrisha Sarkar, Andrei Ioan Muresanu, Carter Blair, Aaryam Sharma, Rakshit S Trivedi, Gillian K Hadfield

TL;DR

The paper tackles enabling cooperative behavior for open-world, generative agents operating under social norms. It introduces the Normative Module, a learning-based architecture that identifies authoritative classification institutions and predicts sanction-based outcomes to align actions accordingly, formalized via a sanction game $\Gamma_S$ embedded in a base game $\Gamma=(N,X,T,A,\Sigma,U)$. A key contribution is the formalization of normative infrastructure through classification institutions that implement a correlated equilibrium $\mathcal{A}_{\mathcal{I}}^{CE}$ to coordinate sanctions, thereby resolving equilibrium selection in cooperation dilemmas. The authors validate their approach with Normative Orchards, demonstrating that normative-module agents converge more rapidly to stable cooperative behavior and achieve higher welfare than baselines, particularly when authoritative institutions are present. This work provides a practical framework for designing generative agents and environments that account for normative structures, with implications for scalable, ethical multi-agent coordination in real-world settings.

Abstract

Generative agents, which implement behaviors using a large language model (LLM) to interpret and evaluate an environment, has demonstrated the capacity to solve complex tasks across many social and technological domains. However, when these agents interact with other agents and humans in presence of social structures such as existing norms, fostering cooperation between them is a fundamental challenge. In this paper, we develop the framework of a 'Normative Module': an architecture designed to enhance cooperation by enabling agents to recognize and adapt to the normative infrastructure of a given environment. We focus on the equilibrium selection aspect of the cooperation problem and inform our agent design based on the existence of classification institutions that implement correlated equilibrium to provide effective resolution of the equilibrium selection problem. Specifically, the normative module enables agents to learn through peer interactions which of multiple candidate institutions in the environment, does a group treat as authoritative. By enabling normative competence in this sense, agents gain ability to coordinate their sanctioning behaviour; coordinated sanctioning behaviour in turn shapes primary behaviour within a social environment, leading to higher average welfare. We design a new environment that supports institutions and evaluate the proposed framework based on two key criteria derived from agent interactions with peers and institutions: (i) the agent's ability to disregard non-authoritative institutions and (ii) the agent's ability to identify authoritative institutions among several options. We show that these capabilities allow the agent to achieve more stable cooperative outcomes compared to baseline agents without the normative module, paving the way for research in a new avenue of designing environments and agents that account for normative infrastructure.

Normative Modules: A Generative Agent Architecture for Learning Norms that Supports Multi-Agent Cooperation

TL;DR

The paper tackles enabling cooperative behavior for open-world, generative agents operating under social norms. It introduces the Normative Module, a learning-based architecture that identifies authoritative classification institutions and predicts sanction-based outcomes to align actions accordingly, formalized via a sanction game embedded in a base game . A key contribution is the formalization of normative infrastructure through classification institutions that implement a correlated equilibrium to coordinate sanctions, thereby resolving equilibrium selection in cooperation dilemmas. The authors validate their approach with Normative Orchards, demonstrating that normative-module agents converge more rapidly to stable cooperative behavior and achieve higher welfare than baselines, particularly when authoritative institutions are present. This work provides a practical framework for designing generative agents and environments that account for normative structures, with implications for scalable, ethical multi-agent coordination in real-world settings.

Abstract

Generative agents, which implement behaviors using a large language model (LLM) to interpret and evaluate an environment, has demonstrated the capacity to solve complex tasks across many social and technological domains. However, when these agents interact with other agents and humans in presence of social structures such as existing norms, fostering cooperation between them is a fundamental challenge. In this paper, we develop the framework of a 'Normative Module': an architecture designed to enhance cooperation by enabling agents to recognize and adapt to the normative infrastructure of a given environment. We focus on the equilibrium selection aspect of the cooperation problem and inform our agent design based on the existence of classification institutions that implement correlated equilibrium to provide effective resolution of the equilibrium selection problem. Specifically, the normative module enables agents to learn through peer interactions which of multiple candidate institutions in the environment, does a group treat as authoritative. By enabling normative competence in this sense, agents gain ability to coordinate their sanctioning behaviour; coordinated sanctioning behaviour in turn shapes primary behaviour within a social environment, leading to higher average welfare. We design a new environment that supports institutions and evaluate the proposed framework based on two key criteria derived from agent interactions with peers and institutions: (i) the agent's ability to disregard non-authoritative institutions and (ii) the agent's ability to identify authoritative institutions among several options. We show that these capabilities allow the agent to achieve more stable cooperative outcomes compared to baseline agents without the normative module, paving the way for research in a new avenue of designing environments and agents that account for normative infrastructure.
Paper Structure (13 sections, 1 equation, 4 figures)

This paper contains 13 sections, 1 equation, 4 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Resolving Cooperation Challenges through Normative Infrastructures
  4. Classification Institutions
  5. Normative modules for reasoning in institutional environments
  6. Experiments
  7. Experiment Setup
  8. Evaluation
  9. Results
  10. Discussion and Future Work
  11. Appendix
  12. Compute
  13. Dialogue Examples

Figures (4)

  • Figure 1: Normative agent architecture for a focal agent. The architecture consists of observations, memory, and normative queries. The agent learns the correct classification institution and uses that classification to select its actions. The outcome of the normative module induces an action transform ($H_{i}$) that resolves the cooperation dilemma.
  • Figure 2: The proportion of actions the agent with the normative module takes that would be classified as acceptable by the non-authoritative classification institution in the experiment with only one non-authoritative classification institution. We vary the number of types of fruit in the environment (y-axis) and the number of background agents in the environment that do not obey the non-authoritative classification institution. When there is only one background agent, following the institution is reasonable because the entire community consists of just you, the non-authoritative institution, and a single community member. Therefore, if we are following the community consensus it is reasonable to follow the institution. When the population size of the background agents increases, the agent with the normative module successfully learns to disobey the non-authoritative classification institution. We repeat all trials 3 times to compute a standard deviation.
  • Figure 3: The proportion of actions the agent with the normative module takes that would be classified as acceptable by the authoritative institution in the experiment with multiple institutions. We vary the total number of institutions (y-axis) and the number of background agents following the authoritative institution (x-axis). The agent with the normative module (right) generally learns to act in accordance with the authoritative institution more than the baseline agent (left). We repeat all trials 3 times to compute a standard deviation.
  • Figure 4: Following the correct Authoritative Institution

Theorems & Definitions (3)

  • Definition 1
  • Definition 2
  • Definition 3