Commute-Time-Optimised Graphs for GNNs

TL;DR

This work tackles the problem of oversquashing in graph neural networks by introducing prior-informed, non-parametric graph rewiring that couples expert interaction priors with sparse, commute-time-optimised expanders. It presents two synthetic priors—salient distance-$d$ interactions and colour-based community interactions—and develops bespoke rewiring schemes (aligned-Cayley expander, distance-$d$-pair rewiring, and colour-Cayley clusters) tailored to these priors. Through synthetic-data experiments and a real-world ogbn-arxiv case study, the authors show regime-specific performance gains, notably faster convergence and improved accuracy in low-data scenarios when priors are effectively integrated. The results highlight the potential of prior-aware rewiring to enhance GNN expressivity and practical deployment, especially in domains where expert knowledge about node interactions is available.

Abstract

We explore graph rewiring methods that optimise commute time. Recent graph rewiring approaches facilitate long-range interactions in sparse graphs, making such rewirings commute-time-optimal on average. However, when an expert prior exists on which node pairs should or should not interact, a superior rewiring would favour short commute times between these privileged node pairs. We construct two synthetic datasets with known priors reflecting realistic settings, and use these to motivate two bespoke rewiring methods that incorporate the known prior. We investigate the regimes where our rewiring improves test performance on the synthetic datasets. Finally, we perform a case study on a real-world citation graph to investigate the practical implications of our work.

Figures (6)

• Figure 1: A base graph (left) and corresponding Cayley clusters rewiring (right). Each node takes one of four colours, or is uncoloured (white). Our Cayley clusters rewiring sparsely connects nodes of the same colour.
• Figure 2: A base graph (left) and corresponding trimmed Cayley expander (right). Nodes are randomly allocated to the base graph nodes; the light grey lines show this random allocation. The three thick edges in the Cayley graph are edges which link nodes at distance 5 in the base graph, which is a prior in one of our synthetic datasets.
• Figure 3: A base graph (left) and corresponding aligned Cayley expander informed by the prior (right). Compared to random Cayley graph placement (Figure \ref{['fig:egp-viz']}), the number of important links captured by the aligned Cayley improves from 3 to 11 (dark edges). The Cayley graphs pictured here and in Figure \ref{['fig:egp-viz']} are isomorphic by construction, i.e. we have only changed how the nodes are aligned with the base graph.
• Figure 4: A sweep of $c_2$/$c_1$ for three values of $c_3$ in Data A, fixing $c_1=1.0$. base-graph-only is the baseline with no rewiring. Each other colour denotes a rewirer introduced by interleaving with the base graph across GNN layers.
• Figure 5: A sweep of $c_2/c_1$ in Data B, fixing $c_1+c_2=1$ (since the target grows very quickly for large $c_2$). base-graph-only is the baseline with no rewiring. Each other colour denotes a rewirer introduced by interleaving with the base graph across GNN layers.