Mixture of Link Predictors on Graphs

TL;DR

This work tackles link prediction by showing that different node pairs benefit from different pairwise signals, challenging the one-size-fits-all approach. It introduces Link-MoE, a mixture-of-experts framework that gates multiple LP predictors using a two-branch gating network informed by local/global heuristics and node-pair features, with predictions computed as $Y_{ij} = \sigma\left(\sum_{o=1}^m G(x_{ij},s_{ij})_o E_o(A,X)_{ij}\right)$. The gating model effectively learns to select appropriate experts per node pair, and a two-step training strategy avoids expert-collapse and enables scalable integration of new predictors. Across eight real-world datasets, including PubMed and ogbl-ppa, Link-MoE substantially outperforms strong baselines and ensemble methods, highlighting the value of adaptive expert selection in LP and suggesting broad applicability to graph-based prediction tasks.

Abstract

Link prediction, which aims to forecast unseen connections in graphs, is a fundamental task in graph machine learning. Heuristic methods, leveraging a range of different pairwise measures such as common neighbors and shortest paths, often rival the performance of vanilla Graph Neural Networks (GNNs). Therefore, recent advancements in GNNs for link prediction (GNN4LP) have primarily focused on integrating one or a few types of pairwise information. In this work, we reveal that different node pairs within the same dataset necessitate varied pairwise information for accurate prediction and models that only apply the same pairwise information uniformly could achieve suboptimal performance. As a result, we propose a simple mixture of experts model Link-MoE for link prediction. Link-MoE utilizes various GNNs as experts and strategically selects the appropriate expert for each node pair based on various types of pairwise information. Experimental results across diverse real-world datasets demonstrate substantial performance improvement from Link-MoE. Notably, Link-MoE achieves a relative improvement of 18.71\% on the MRR metric for the Pubmed dataset and 9.59\% on the Hits@100 metric for the ogbl-ppa dataset, compared to the best baselines.

Figures (14)

• Figure 1: Hits@K of combined heuristics.
• Figure 2: The overlapping ratio of heuristics.
• Figure 3: The overlapping ratio on ogbl-collab.
• Figure 4: The performance of different models on each CN group.
• Figure 5: An overview of the proposed Link-MoE.