Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PEnGUiN: Partially Equivariant Graph NeUral Networks for Sample Efficient MARL

Joshua McClellan, Greyson Brothers, Furong Huang, Pratap Tokekar

TL;DR

PEnGUiN introduces Partially Equivariant Graph Neural Networks, a flexible MARL architecture that blends fully equivariant and non-equivariant updates through a learnable symmetry score $\alpha$, enabling adaptation to partial symmetries. It formalizes four partial-equivariance categories and provides a unified framework that encompasses both EGNN and GNN representations; an Equivariance Estimator allows spatially and entity-aware modulation of $\alpha$. Empirical results on modified Multi-Particle Environments and highway-env show PEnGUiN outperforms both fully equivariant and non-equivariant baselines under asymmetries, demonstrating improved sample efficiency and robustness. The work advances the applicability of symmetry-based inductive biases to real-world MARL problems by handling regional, feature-wise, and approximate symmetries, with potential impact on autonomous driving and multi-robot systems.

Abstract

Equivariant Graph Neural Networks (EGNNs) have emerged as a promising approach in Multi-Agent Reinforcement Learning (MARL), leveraging symmetry guarantees to greatly improve sample efficiency and generalization. However, real-world environments often exhibit inherent asymmetries arising from factors such as external forces, measurement inaccuracies, or intrinsic system biases. This paper introduces \textit{Partially Equivariant Graph NeUral Networks (PEnGUiN)}, a novel architecture specifically designed to address these challenges. We formally identify and categorize various types of partial equivariance relevant to MARL, including subgroup equivariance, feature-wise equivariance, regional equivariance, and approximate equivariance. We theoretically demonstrate that PEnGUiN is capable of learning both fully equivariant (EGNN) and non-equivariant (GNN) representations within a unified framework. Through extensive experiments on a range of MARL problems incorporating various asymmetries, we empirically validate the efficacy of PEnGUiN. Our results consistently demonstrate that PEnGUiN outperforms both EGNNs and standard GNNs in asymmetric environments, highlighting their potential to improve the robustness and applicability of graph-based MARL algorithms in real-world scenarios.

PEnGUiN: Partially Equivariant Graph NeUral Networks for Sample Efficient MARL

TL;DR

PEnGUiN introduces Partially Equivariant Graph Neural Networks, a flexible MARL architecture that blends fully equivariant and non-equivariant updates through a learnable symmetry score , enabling adaptation to partial symmetries. It formalizes four partial-equivariance categories and provides a unified framework that encompasses both EGNN and GNN representations; an Equivariance Estimator allows spatially and entity-aware modulation of . Empirical results on modified Multi-Particle Environments and highway-env show PEnGUiN outperforms both fully equivariant and non-equivariant baselines under asymmetries, demonstrating improved sample efficiency and robustness. The work advances the applicability of symmetry-based inductive biases to real-world MARL problems by handling regional, feature-wise, and approximate symmetries, with potential impact on autonomous driving and multi-robot systems.

Abstract

Equivariant Graph Neural Networks (EGNNs) have emerged as a promising approach in Multi-Agent Reinforcement Learning (MARL), leveraging symmetry guarantees to greatly improve sample efficiency and generalization. However, real-world environments often exhibit inherent asymmetries arising from factors such as external forces, measurement inaccuracies, or intrinsic system biases. This paper introduces \textit{Partially Equivariant Graph NeUral Networks (PEnGUiN)}, a novel architecture specifically designed to address these challenges. We formally identify and categorize various types of partial equivariance relevant to MARL, including subgroup equivariance, feature-wise equivariance, regional equivariance, and approximate equivariance. We theoretically demonstrate that PEnGUiN is capable of learning both fully equivariant (EGNN) and non-equivariant (GNN) representations within a unified framework. Through extensive experiments on a range of MARL problems incorporating various asymmetries, we empirically validate the efficacy of PEnGUiN. Our results consistently demonstrate that PEnGUiN outperforms both EGNNs and standard GNNs in asymmetric environments, highlighting their potential to improve the robustness and applicability of graph-based MARL algorithms in real-world scenarios.

Paper Structure

This paper contains 19 sections, 2 theorems, 9 equations, 14 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Background
  4. Multi-Agent Reinforcement Learning
  5. Equivariance
  6. Partially Equivariant Graph Neural Networks
  7. Categories of Partial Equivariance
  8. Experiments
  9. Multi-Particle Environment (MPE)
  10. Results and Discussion (MPE)
  11. PEnGUiN Quantifying Partial Equivariance
  12. Experiments on Highway Environment
  13. Environment Setup
  14. Results and Discussion (highway-env)
  15. Conclusion
  16. ...and 4 more sections

Key Result

Theorem 1

Given a Partially Equivariant Graph Neural Network Layer as defined in table tab:penguin_equations_compact, when $\alpha = 1$ the Partial Equivariant Layer is exactly equivalent to an E2GN2 layer. (see Appendix A for proof)

Figures (14)

  • Figure 1: An example of how EGNNs can be advantageous in equivariant environments, and a liability when an environment has increased asymmetries.
  • Figure 2: Graph at layer $l$
  • Figure 3: Node Feature Update $\ {\bm{h}}_i^{l+1}$
  • Figure 4: Node Coordinate Update $\ {\bm{u}}_i^{l+1}$
  • Figure 6: External Forces
  • ...and 9 more figures

Theorems & Definitions (6)

  • Theorem 1
  • Theorem 2
  • Definition 5.1: Approximate Equivariance
  • Definition 5.2: Subgroup Equivariance
  • Definition 5.3: Feature-wise equivariance
  • Definition 5.4: Regional Equivariance