Invariant and Equivariant Graph Networks

TL;DR

The paper delivers a complete, size-agnostic characterization of permutation-invariant and -equivariant linear layers for graph and hyper-graph data. It shows that invariant layers on k-way tensors have dimension $\mathrm{b}(k)$ and equivariant layers to $\mathbb{R}^{n^{l}}$ have dimension $\mathrm{b}(k+l)$, with orthogonal bases indexed by equality-pattern partitions, enabling maximal expressive power. The framework extends to biases, node features, and mixed-order layers, and generalizes to multiple node-sets, yielding scalable, universal-approximation power (at least as strong as MPNNs) for graph learning tasks. Empirical results demonstrate competitive performance on graph classification and the ability to extrapolate to different graph sizes, while also outperforming existing invariant bases in expressive tasks. Overall, the work unifies and extends invariant/equivariant graph learning, offering practical, efficient layer constructions with strong theoretical guarantees.

Abstract

Invariant and equivariant networks have been successfully used for learning images, sets, point clouds, and graphs. A basic challenge in developing such networks is finding the maximal collection of invariant and equivariant linear layers. Although this question is answered for the first three examples (for popular transformations, at-least), a full characterization of invariant and equivariant linear layers for graphs is not known. In this paper we provide a characterization of all permutation invariant and equivariant linear layers for (hyper-)graph data, and show that their dimension, in case of edge-value graph data, is 2 and 15, respectively. More generally, for graph data defined on k-tuples of nodes, the dimension is the k-th and 2k-th Bell numbers. Orthogonal bases for the layers are computed, including generalization to multi-graph data. The constant number of basis elements and their characteristics allow successfully applying the networks to different size graphs. From the theoretical point of view, our results generalize and unify recent advancement in equivariant deep learning. In particular, we show that our model is capable of approximating any message passing neural network Applying these new linear layers in a simple deep neural network framework is shown to achieve comparable results to state-of-the-art and to have better expressivity than previous invariant and equivariant bases.

Figures (2)

• Figure 1: The full basis for equivariant linear layers for edge-value data ${\bm{\mathsfit{A}}}\in\mathbb R^{n\times n}$, for $n=5$. The purely linear $15$ basis elements, ${\bm{\mathsfit{B}}}^\mu$, are represented by matrices $n^2\times n^2$, and the $2$ bias basis elements (right), ${\bm{\mathsfit{C}}}^\lambda$, by matrices $n\times n$, see equation \ref{['e:basis_general']}.
• Figure :

Theorems & Definitions (14)

• Proposition 1
• proof
• Proposition 2
• proof
• Theorem 1
• Theorem 2
• Theorem 3
• Proposition 3
• proof
• proof