The Effectiveness of Curvature-Based Rewiring and the Role of Hyperparameters in GNNs Revisited

TL;DR

This work reevaluate the effectiveness of curvature-based rewiring in real-world datasets and nuances the effectiveness of curvature-based rewiring in real-world datasets and brings a new perspective on the methods to evaluate GNN accuracy improvements.

Abstract

Message passing is the dominant paradigm in Graph Neural Networks (GNNs). The efficiency of message passing, however, can be limited by the topology of the graph. This happens when information is lost during propagation due to being oversquashed when travelling through bottlenecks. To remedy this, recent efforts have focused on graph rewiring techniques, which disconnect the input graph originating from the data and the computational graph, on which message passing is performed. A prominent approach for this is to use discrete graph curvature measures, of which several variants have been proposed, to identify and rewire around bottlenecks, facilitating information propagation. While oversquashing has been demonstrated in synthetic datasets, in this work we reevaluate the performance gains that curvature-based rewiring brings to real-world datasets. We show that in these datasets, edges selected during the rewiring process are not in line with theoretical criteria identifying bottlenecks. This implies they do not necessarily oversquash information during message passing. Subsequently, we demonstrate that SOTA accuracies on these datasets are outliers originating from sweeps of hyperparameters -- both the ones for training and dedicated ones related to the rewiring algorithm -- instead of consistent performance gains. In conclusion, our analysis nuances the effectiveness of curvature-based rewiring in real-world datasets and brings a new perspective on the methods to evaluate GNN accuracy improvements.

Figures (17)

• Figure 1: A visualisation of the edges selected during the SDRF rewiring algorithm. a, The panels show the edges that do not satisfy condition 2b, both due to $\delta > 1/\sharp_{\triangle}$ (if the edge is situated above the dotted line) or $\delta > 1/\gamma_{max}$ (if the edge is situated below the dotted line). b, The panels show the opposite, namely the edges that satisfy condition 2b. This means that the edge is situated below the dotted line and $\delta < 1/\gamma_{max}$. The color code of the edges indicates at which step of the rewiring process (in %) the edge is selected. Dotted line shows $y = 1/\sharp_{\triangle}$ corresponding to the upper limit in condition 2b.
• Figure 2: Distribution of mean test accuracy over the sweep of hyperparameters for the different curvature measures and node-classification datasets used. We show boxenplots which first identify the median, then extend boxes outward, each covering half of the remaining data on which outliers (circles) are identifiable. For each dataset we also show the smoothed distribution using kernel density estimates from the seaborn package.
• Figure 3: Distribution of curvatures for dataset Texas
• Figure 4: Distribution of curvatures for dataset Cornell
• Figure 5: Distribution of curvatures for dataset Wisconsin