Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Operationalising Rawlsian Ethics for Fairness in Norm-Learning Agents

Jessica Woodgate, Paul Marshall, Nirav Ajmeri

TL;DR

The paper addresses the emergence of fair norms in multi-agent systems by integrating normative ethics into learning agents. It introduces RAWL-${\cdot}$E, a framework that operationalises Rawlsian maximin through an ethics module that shapes rewards, enabling explicit norm learning via a norms module and learning via a DQN-based interaction module. The key contributions are the design of an ethics-informed norm-learning architecture, the demonstration that emergent norms prioritize the least advantaged, and empirical evidence showing improved social welfare, reduced inequality, and greater robustness in two harvesting scenarios. The results suggest that embedding normative ethics into RL agents can yield fairer, more sustainable, and cooperative MAS with practical implications for ethically-aligned autonomous systems.

Abstract

Social norms are standards of behaviour common in a society. However, when agents make decisions without considering how others are impacted, norms can emerge that lead to the subjugation of certain agents. We present RAWL-E, a method to create ethical norm-learning agents. RAWL-E agents operationalise maximin, a fairness principle from Rawlsian ethics, in their decision-making processes to promote ethical norms by balancing societal well-being with individual goals. We evaluate RAWL-E agents in simulated harvesting scenarios. We find that norms emerging in RAWL-E agent societies enhance social welfare, fairness, and robustness, and yield higher minimum experience compared to those that emerge in agent societies that do not implement Rawlsian ethics.

Operationalising Rawlsian Ethics for Fairness in Norm-Learning Agents

TL;DR

The paper addresses the emergence of fair norms in multi-agent systems by integrating normative ethics into learning agents. It introduces RAWL-E, a framework that operationalises Rawlsian maximin through an ethics module that shapes rewards, enabling explicit norm learning via a norms module and learning via a DQN-based interaction module. The key contributions are the design of an ethics-informed norm-learning architecture, the demonstration that emergent norms prioritize the least advantaged, and empirical evidence showing improved social welfare, reduced inequality, and greater robustness in two harvesting scenarios. The results suggest that embedding normative ethics into RL agents can yield fairer, more sustainable, and cooperative MAS with practical implications for ethically-aligned autonomous systems.

Abstract

Social norms are standards of behaviour common in a society. However, when agents make decisions without considering how others are impacted, norms can emerge that lead to the subjugation of certain agents. We present RAWL-E, a method to create ethical norm-learning agents. RAWL-E agents operationalise maximin, a fairness principle from Rawlsian ethics, in their decision-making processes to promote ethical norms by balancing societal well-being with individual goals. We evaluate RAWL-E agents in simulated harvesting scenarios. We find that norms emerging in RAWL-E agent societies enhance social welfare, fairness, and robustness, and yield higher minimum experience compared to those that emerge in agent societies that do not implement Rawlsian ethics.

Paper Structure

This paper contains 43 sections, 4 equations, 7 figures, 8 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Contribution
  3. Novelty
  4. Organisation
  5. Related Works
  6. Normative Ethics and Norm Emergence
  7. Normative Ethics and Learning
  8. Method
  9. Schematic
  10. Interaction and Norm Learning
  11. Ethics Module
  12. Norms Module
  13. Interaction Module
  14. Simulation Environment
  15. Scenario
  16. ...and 28 more sections

Figures (7)

  • Figure 1: Harvesting environment. (a) Capabilities harvest scenario explores how agents learn to identify and reach desired berries while considering the well-being of the society. (b) Allotment harvest scenario explores how agents learn to harvest within their desired areas while considering the well-being in the society.
  • Figure 2: Comparing Gini index of $ag\textsubscript{well-being}$ and $ag\textsubscript{resource}$ for $e$. Lower Gini in RAWL${\cdot}$E indicates lower inequality.
  • Figure 3: Minimum $ag\textsubscript{well-being}$ over $t\textsubscript{max}$ steps summed for $e$, normalised by step frequency. RAWL${\cdot}$E yields higher minimum well-being.
  • Figure 4: Cumulative $ag\textsubscript{well-being}$ and $ag\textsubscript{resource}$ of each society over $t\textsubscript{max}$ steps summed for $e$, normalised by step frequency. Societies of RAWL${\cdot}$E agents have higher well-being and cumulative resource consumption.
  • Figure 5: Days survived for $e$. Societies of RAWL${\cdot}$E agents survive for longer, indicating higher robustness.
  • ...and 2 more figures

Theorems & Definitions (7)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7