Spectral Theory for Edge Pruning in Asynchronous Recurrent Graph Neural Networks

TL;DR

This work addresses the efficiency challenges of asynchronous recurrent graph neural networks (ARGNNs) by introducing a dynamic, decentralized pruning method grounded in graph spectral theory. The core idea leverages the imaginary parts of local Laplacian eigenvalues, $\Im(\lambda)$, to identify edge redundancy and prune edges where two nodes exhibit opposite imaginary components, implemented via local Laplacians $L_n = D_n - A_n \odot W_n$. The study compares two degree configurations, $D$ and $D_W$, across logic-gate tasks (AND, OR, XOR), showing that pruning with $D_W$ yields stronger edge compression with minimal accuracy loss, and that pruning timing differs between configurations. While promising, the approach remains preliminary, omitting eigenvectors and real parts, and invites future work to capture higher-order couplings and richer spectral features to further enhance pruning in larger decentralized networks.

Abstract

Graph Neural Networks (GNNs) have emerged as a powerful tool for learning on graph-structured data, finding applications in numerous domains including social network analysis and molecular biology. Within this broad category, Asynchronous Recurrent Graph Neural Networks (ARGNNs) stand out for their ability to capture complex dependencies in dynamic graphs, resembling living organisms' intricate and adaptive nature. However, their complexity often leads to large and computationally expensive models. Therefore, pruning unnecessary edges becomes crucial for enhancing efficiency without significantly compromising performance. This paper presents a dynamic pruning method based on graph spectral theory, leveraging the imaginary component of the eigenvalues of the network graph's Laplacian.

Figures (4)

• Figure 1: Example for an XOR gate; the edges $4 \rightarrow 3$ and $3 \rightarrow 2$ were removed during training.
• Figure 2: Development of the real and imaginary components of the eigenvalues shown for each node for the Degree mode $D$. Nodes $n_0$ and $n_1$ are excluded as they serve as placeholders for input values and do not participate in any behavior. Each color corresponds to the component value associated with a particular node. For clarity, consider node $n_2$: it prunes its input edge from $n_3$ at the moment this is detected (indicated by the legend $\triangle n_2 P(3 \rightarrow 2)$). This decoupling enhances the dynamic $n_2 \rightarrow n_3$, which cannot be pruned by $n_2$ since nodes are restricted to pruning only their input edges. Later in the optimization process, $n_3$ prunes its input edge from $n_4$, marked by $\square n_3 P(4 \rightarrow 3)$. The effect of this decoupling is visible in the neighborhood dynamics of $n_4$, where the imaginary component of $n_4$, initially zero, becomes coupled with $n_3$, in particular, the coupling appears in the edge $n_3 \rightarrow n_4$.
• Figure 3: Experiment results: mean and standard deviation of $10$ runs per number of hidden nodes. The evaluated logic gates are shown with their respective after-training errors in the left column, for both Degree matrix modes directed $D$ and weighted directed $D_{W}$, $None$ indicates the unpruned network acting as a baseline for the error. The number of edges removed for different numbers of hidden nodes and both degree modes is shown in the right column in both absolute and percentual values.
• Figure 4: Mean and standard deviation of error and percentage of pruned edges during training for $5$ runs of a network with $2$ hidden nodes trained on the $XOR$ logic gate task.