Repetition Makes Perfect: Recurrent Graph Neural Networks Match Message-Passing Limit

TL;DR

This work establishes that computable recurrent graph neural networks with finite-precision weights and ReLU activations can uniformly express all mp-invariant graph computations, matching the color-refinement (1-WL) limit with only polynomial time and space overhead. Introducing random node initialization elevates this power to all graph algorithms on connected graphs, effectively enabling universal computation for graphs in polynomial time. The authors develop a rigorous reduction chain from mp-invariant computations to R-GNNs, including encodings and a switched recurrent architecture that emulates Turing machines within a sum-aggregation framework. They also explore graph embeddings and WL-invariance under global sum, clarifying the boundaries between mp-invariance and WL-invariance and highlighting practical implications for the design of powerful recurrent graph architectures.

Abstract

We precisely characterize the expressivity of computable Recurrent Graph Neural Networks (recurrent GNNs). We prove that recurrent GNNs with finite-precision parameters, sum aggregation, and ReLU activation, can compute any graph algorithm that respects the natural message-passing invariance induced by the Color Refinement (or Weisfeiler-Leman) algorithm. While it is well known that the expressive power of GNNs is limited by this invariance [Morris et al., AAAI 2019; Xu et al., ICLR 2019], we establish that recurrent GNNs can actually match this limit. This is in contrast to non-recurrent GNNs, which have the power of Weisfeiler-Leman only in a very weak, "non-uniform", sense where each graph size requires a different GNN to compute with. Our construction introduces only a polynomial overhead in both time and space. Furthermore, we show that by incorporating random initialization, for connected graphs recurrent GNNs can express all graph algorithms. In particular, any polynomial-time graph algorithm can be emulated on connected graphs in polynomial time by a recurrent GNN with random initialization.

Figures (5)

• Figure 1: Uniform expressivity hierarchy. 'CR' and 'WL' are acronyms for Color Refinement and Weisfeiler-Leman. The results in this paper are the equivalencies. 'Node Functions' means computable functions $f(G,v)$ operating on a graph and node.
• Figure 2: An overview of the reduction sequence from message-passing algorithms and mp-invariant functions to R-GNNs. Every message-passing algorithm is mp-invariant - by induction on the number of iterations. Then, starting from the mp-invariant class and moving clockwise, the reductions correspond to \ref{['lem:crinv']}; Lemma \ref{['lem:mpcgc_to_mpgc']}; Lemma \ref{['lem:mpgc_to_mpgc']}; and Lemma \ref{['lem:smpgc_to_rgnn']}.
• Figure 3: Two rounds of Color Refinement on a graph $G$ shown in (a). Colors can be represented as trees (b) or dags (c). The actual colors in the figure illustrate the coloring reached after two steps (they do not indicate initial features of the nodes).
• Figure 4: Let $G\in{\mathcal{G}}_{{\mathcal{B}}_k}$ and define $n\coloneqq |G|$. An illustration for reducing the construction of $\mathsf{mpc}_G(v)$ to R-GNN. First it is reduced to $O(n)$ iterations of MP-LGA. Then, each iteration of MP-LGA is reduced to $O(n^3)$ iterations of S-MP-GA. Let that S-MP-GA be $S$, and let $T_S(G,i)$ be the number of Turing machine steps required to compute the $i^{th}$ iteration of $S$ when operating on $G$. Then, iteration $i,i\in[2n^4]$ is reduced to $O(I_S(G,i))$ recurrences of an R-GNN, where $I_S(G,i)\coloneqq T_S(G,i)+(n^3\log(n)+kn)^2$. The $(n^3\log(n)+kn)^2$ overhead is for translating the sum of messages from RB to RQ.
• Figure 5: The structure of $F$. In yellow are all the switched sub-networks that pass control and data from one to another. In green is the synchronization sub-network that runs continuously. In beige and blue are the dimensions that assume the sum of neighbors' sub-networks output in the previous recurrence. Layer $y'$ is the outputs of the individual sub-networks. These, together with the neighbors' sum, are inter-routed using layers $[y'+1..y]$, to create the desired interoperability. Finally, the output of layer $y$ is sent to neighbors and also becomes (half) the input for the next recurrence.

Theorems & Definitions (68)

• Definition 3.1
• Lemma 3.2
• Definition 4.1
• Theorem 4.2
• Definition 4.3
• Definition 4.4
• Lemma 4.4
• Definition 4.5
• Lemma 4.5
• Lemma 4.5