Universal Link Predictor By In-Context Learning on Graphs

TL;DR

This paper tackles the problem of transferring link-prediction capability across diverse graphs. It introduces UniLP, a universal link predictor that uses in-context learning and attention over ego-subgraphs to condition predictions on the target graph without explicit training on that graph, effectively modeling $p(1|A,c)$. By combining the generalizability of heuristic insights with data-driven pattern learning, UniLP achieves competitive or superior performance to pretrained and even some finetuned GNN-based LP models across multiple unseen graphs. The work demonstrates robust cross-graph adaptability and highlights the practical potential of in-context graph learning for scalable, training-free link prediction in real-world networks.

Abstract

Link prediction is a crucial task in graph machine learning, where the goal is to infer missing or future links within a graph. Traditional approaches leverage heuristic methods based on widely observed connectivity patterns, offering broad applicability and generalizability without the need for model training. Despite their utility, these methods are limited by their reliance on human-derived heuristics and lack the adaptability of data-driven approaches. Conversely, parametric link predictors excel in automatically learning the connectivity patterns from data and achieving state-of-the-art but fail short to directly transfer across different graphs. Instead, it requires the cost of extensive training and hyperparameter optimization to adapt to the target graph. In this work, we introduce the Universal Link Predictor (UniLP), a novel model that combines the generalizability of heuristic approaches with the pattern learning capabilities of parametric models. UniLP is designed to autonomously identify connectivity patterns across diverse graphs, ready for immediate application to any unseen graph dataset without targeted training. We address the challenge of conflicting connectivity patterns-arising from the unique distributions of different graphs-through the implementation of In-context Learning (ICL). This approach allows UniLP to dynamically adjust to various target graphs based on contextual demonstrations, thereby avoiding negative transfer. Through rigorous experimentation, we demonstrate UniLP's effectiveness in adapting to new, unseen graphs at test time, showcasing its ability to perform comparably or even outperform parametric models that have been finetuned for specific datasets. Our findings highlight UniLP's potential to set a new standard in link prediction, combining the strengths of heuristic and parametric methods in a single, versatile framework.

Figures (8)

• Figure 1: Performance change of SEAL zhang_link_2018 after training with one additional graph. denotes statistically significant change.
• Figure 2: Two synthetic graphs with different connectivity patterns: (a) Grid lattice graph; (b) Triangular lattice graph.
• Figure 3: Overview of the Universal Link Predictor framework. (a) For predicting a query link $q$, we initially sample positive ($s^+$) and negative ($s^-$) in-context links from the target graph. Both the query link and these in-context links are independently processed through a shared subgraph GNN encoder. An attention mechanism then calculates scores based on the similarity between the query link and the in-context links. (b) The final representation of the query link, contextualized by the target graph, is obtained through a weighted summation, which combines the representations of the in-context links with their respective labels.
• Figure 4: Performance of UniLP with varying quantities of in-context links.
• Figure 5: Visualization of the link representation learned from (a) Pretrain Only SEAL; (b) UniLP. Different colors indicate different test datasets.

Theorems & Definitions (2)

• Definition 2.1
• Theorem 2.2