Predict then Propagate: Graph Neural Networks meet Personalized PageRank

TL;DR

This work tackles semi-supervised node classification on graphs by decoupling the prediction and propagation stages and leveraging personalized PageRank to enable a large, adjustable neighborhood without oversmoothing. It introduces PPNP, which propagates node-wise predictions generated by a neural network through a personalized PageRank operator, and APPNP, a fast linear-time approximation using power iterations. The approach achieves state-of-the-art accuracy with fewer parameters and similar or faster training times across multiple graph benchmarks, validated by a rigorous evaluation protocol. The results demonstrate that large-range, locality-preserving propagation can substantially improve performance and can be combined with arbitrary predictors, offering a flexible and scalable framework for graph-based learning.

Abstract

Neural message passing algorithms for semi-supervised classification on graphs have recently achieved great success. However, for classifying a node these methods only consider nodes that are a few propagation steps away and the size of this utilized neighborhood is hard to extend. In this paper, we use the relationship between graph convolutional networks (GCN) and PageRank to derive an improved propagation scheme based on personalized PageRank. We utilize this propagation procedure to construct a simple model, personalized propagation of neural predictions (PPNP), and its fast approximation, APPNP. Our model's training time is on par or faster and its number of parameters on par or lower than previous models. It leverages a large, adjustable neighborhood for classification and can be easily combined with any neural network. We show that this model outperforms several recently proposed methods for semi-supervised classification in the most thorough study done so far for GCN-like models. Our implementation is available online.

Figures (13)

• Figure 1: Illustration of (approximate) personalized propagation of neural predictions (PPNP, APPNP). Predictions are first generated from each node's own features by a neural network and then propagated using an adaptation of personalized PageRank. The model is trained end-to-end.
• Figure 2: Accuracy distributions of different models. The high standard deviation between data splits and initializations shows the importance of a rigorous evaluation, which is often omitted.
• Figure 3: Accuracy for different training set sizes (number of labeled nodes per class) on Cora-ML. PPNP's dominance increases further for smaller training set sizes.
• Figure 4: Accuracy depending on the number of propagation steps $K$. The accuracy breaks down for the GCN-like propagation ($\alpha=0$), while it increases and stabilizes when using APPNP ($\alpha=0.1$).
• Figure 5: Accuracy depending on teleport probability $\alpha$. The optimum typically lies within $\alpha \in [0.05, 0.2]$, but changes for different types of datasets.