Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Revisiting Multi-Permutation Equivariance through the Lens of Irreducible Representations

Yonatan Sverdlov, Ido Springer, Nadav Dym

Abstract

This paper explores the characterization of equivariant linear layers for representations of permutations and related groups. Unlike traditional approaches, which address these problems using parameter-sharing, we consider an alternative methodology based on irreducible representations and Schur's lemma. Using this methodology, we obtain an alternative derivation for existing models like DeepSets, 2-IGN graph equivariant networks, and Deep Weight Space (DWS) networks. The derivation for DWS networks is significantly simpler than that of previous results. Next, we extend our approach to unaligned symmetric sets, where equivariance to the wreath product of groups is required. Previous works have addressed this problem in a rather restrictive setting, in which almost all wreath equivariant layers are Siamese. In contrast, we give a full characterization of layers in this case and show that there is a vast number of additional non-Siamese layers in some settings. We also show empirically that these additional non-Siamese layers can improve performance in tasks like graph anomaly detection, weight space alignment, and learning Wasserstein distances. Our code is available at \href{https://github.com/yonatansverdlov/Irreducible-Representations-of-Deep-Weight-Spaces}{GitHub}.

Revisiting Multi-Permutation Equivariance through the Lens of Irreducible Representations

Abstract

This paper explores the characterization of equivariant linear layers for representations of permutations and related groups. Unlike traditional approaches, which address these problems using parameter-sharing, we consider an alternative methodology based on irreducible representations and Schur's lemma. Using this methodology, we obtain an alternative derivation for existing models like DeepSets, 2-IGN graph equivariant networks, and Deep Weight Space (DWS) networks. The derivation for DWS networks is significantly simpler than that of previous results. Next, we extend our approach to unaligned symmetric sets, where equivariance to the wreath product of groups is required. Previous works have addressed this problem in a rather restrictive setting, in which almost all wreath equivariant layers are Siamese. In contrast, we give a full characterization of layers in this case and show that there is a vast number of additional non-Siamese layers in some settings. We also show empirically that these additional non-Siamese layers can improve performance in tasks like graph anomaly detection, weight space alignment, and learning Wasserstein distances. Our code is available at \href{https://github.com/yonatansverdlov/Irreducible-Representations-of-Deep-Weight-Spaces}{GitHub}.

Paper Structure

This paper contains 35 sections, 10 theorems, 72 equations, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Learning Sets of Symmetric Elements.
  4. Equivariant Characterization Via Irreducibles.
  5. preliminaries
  6. Schur's lemma.
  7. Layer characterization using Schur's Lemma
  8. Computing linear equivariant layers
  9. Deep Sets
  10. Equivariant graph layers
  11. Deep weight spaces
  12. Sets of unaligned symmetric elements
  13. Wreath-equivariant layers.
  14. Examples.
  15. Beyond $S_k$.
  16. ...and 20 more sections

Key Result

Theorem 4.1

For all $n\geq 4$, the space $\mathbb{R}^{n\times n}$ can be written as a direct sum of the spaces $\mathcal{V}_0,\ldots,\mathcal{V}_6$. These spaces are invariant and irreducible, and the isomorphism relations between them are given by $\mathcal{V}_0 \cong \mathcal{V}_1, \mathcal{V}_2\cong \mathcal

Theorems & Definitions (16)

  • Theorem 4.1
  • Theorem 4.2
  • proof
  • Theorem 5.1
  • proof
  • Theorem 5.2
  • Lemma C.1
  • proof : Proof of the Lemma.
  • Theorem C.1
  • proof
  • ...and 6 more