Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pure Message Passing Can Estimate Common Neighbor for Link Prediction

Kaiwen Dong, Zhichun Guo, Nitesh V. Chawla

TL;DR

This work studies the proficiency of MPNNs in approximating CN heuristics, and introduces the Message Passing Link Predictor (MPLP), a novel link prediction model that taps into quasi-orthogonal vectors to estimate link-level structural features, all while preserving the node-level complexities.

Abstract

Message Passing Neural Networks (MPNNs) have emerged as the {\em de facto} standard in graph representation learning. However, when it comes to link prediction, they often struggle, surpassed by simple heuristics such as Common Neighbor (CN). This discrepancy stems from a fundamental limitation: while MPNNs excel in node-level representation, they stumble with encoding the joint structural features essential to link prediction, like CN. To bridge this gap, we posit that, by harnessing the orthogonality of input vectors, pure message-passing can indeed capture joint structural features. Specifically, we study the proficiency of MPNNs in approximating CN heuristics. Based on our findings, we introduce the Message Passing Link Predictor (MPLP), a novel link prediction model. MPLP taps into quasi-orthogonal vectors to estimate link-level structural features, all while preserving the node-level complexities. Moreover, our approach demonstrates that leveraging message-passing to capture structural features could offset MPNNs' expressiveness limitations at the expense of estimation variance. We conduct experiments on benchmark datasets from various domains, where our method consistently outperforms the baseline methods.

Pure Message Passing Can Estimate Common Neighbor for Link Prediction

TL;DR

This work studies the proficiency of MPNNs in approximating CN heuristics, and introduces the Message Passing Link Predictor (MPLP), a novel link prediction model that taps into quasi-orthogonal vectors to estimate link-level structural features, all while preserving the node-level complexities.

Abstract

Message Passing Neural Networks (MPNNs) have emerged as the {\em de facto} standard in graph representation learning. However, when it comes to link prediction, they often struggle, surpassed by simple heuristics such as Common Neighbor (CN). This discrepancy stems from a fundamental limitation: while MPNNs excel in node-level representation, they stumble with encoding the joint structural features essential to link prediction, like CN. To bridge this gap, we posit that, by harnessing the orthogonality of input vectors, pure message-passing can indeed capture joint structural features. Specifically, we study the proficiency of MPNNs in approximating CN heuristics. Based on our findings, we introduce the Message Passing Link Predictor (MPLP), a novel link prediction model. MPLP taps into quasi-orthogonal vectors to estimate link-level structural features, all while preserving the node-level complexities. Moreover, our approach demonstrates that leveraging message-passing to capture structural features could offset MPNNs' expressiveness limitations at the expense of estimation variance. We conduct experiments on benchmark datasets from various domains, where our method consistently outperforms the baseline methods.
Paper Structure (63 sections, 3 theorems, 34 equations, 10 figures, 14 tables)

This paper contains 63 sections, 3 theorems, 34 equations, 10 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and Related Work
  3. Notations.
  4. Link predictions.
  5. GNNs for link prediction.
  6. Can Message Passing count Common Neighbor?
  7. Estimation via Mean Squared Error Regression
  8. Estimation capabilities of GNNs for link predictors
  9. Quasi-orthogonal vectors.
  10. Limitations.
  11. Unbiased estimator.
  12. Orthogonal node attributes.
  13. Multi-layer message passing
  14. Method
  15. Node representation.
  16. ...and 48 more sections

Key Result

Theorem 3.1

Let $G = (V, E)$ be a non-attributed graph and consider a 1-layer GCN/SAGE. Define the input vectors ${\bm{X}} \in \mathbb{R}^{N\times F}$ initialized randomly from a zero-mean distribution with standard deviation $\sigma_{node}$. Additionally, let the weight matrix ${\bm{W}} \in \mathbb{R}^{F^{\pri where $\hat{d}_v = {d}_v + 1$ and $C = \sigma^2_{node}\sigma^2_{weight}FF^\prime$.

Figures (10)

  • Figure 1: (a) Isomorphic nodes result in identical MPNN node representation, making it impossible to distinguish links such as $(v_1,v_3)$ and $(v_1,v_5)$ based on these representations. (b) MPNN counts Common Neighbor through the inner product of neighboring nodes' one-hot representation.
  • Figure 2: GNNs estimate CN, AA and RA via MSE regression, using the mean value as a Baseline. Lower values are better.
  • Figure 3: Representation of the target link $(u,v)$ within our model (MPLP), with nodes color-coded based on their distance from the target link.
  • Figure 4: Evaluation of inference time on large-scale OGB datasets. The inference time encompasses the entire cycle within a full-batch inference.
  • Figure 4: Performance of different GNNs on learning the counts of triangles, measured by MSE divided by variance of the ground truth counts. Shown here are the median (i.e., third-best) performances of each model over five runs with different random seeds.
  • ...and 5 more figures

Theorems & Definitions (6)

  • Theorem 3.1
  • Theorem 3.2
  • Theorem 3.3
  • proof
  • proof
  • proof