Towards Graph Foundation Models: Training on Knowledge Graphs Enables Transferability to General Graphs

TL;DR

SCR proposes a unified KG-based pretraining objective for graph foundation models, enabling zero-shot transfer from knowledge graphs to diverse general graphs. It introduces a Semantic Conditional Message Passing framework that fuses local semantic context with global topology via a semantic feature unifier and a novel Init^2 mechanism, while reformulating node/graph tasks as KG reasoning using task-specific KG structures. Empirical results across 38 datasets show SCR outperforms prior zero-shot baselines and is robust to varying semantic spaces, with strong gains on node and graph classification and meaningful ablations demonstrating the value of semantic integration. This approach offers a scalable path toward versatile, transferable graph representations applicable across domains without task-specific fine-tuning.

Abstract

Inspired by the success of large language models, there is a trend toward developing graph foundation models to conduct diverse downstream tasks in various domains. However, current models often require extra fine-tuning to apply their learned structural and semantic representations to new graphs, which limits their versatility. Recent breakthroughs in zero-shot inductive reasoning on knowledge graphs (KGs), offer us a new perspective on extending KG reasoning to general graph applications. In this paper, we introduce SCR, a unified graph reasoning framework designed to train on knowledge graphs and effectively generalize across a wide range of graph tasks and domains. We begin by designing the task-specific KG structures to establish a unified topology for different task formats. Then we propose semantic-conditioned message passing, a novel mechanism addressing the inherent semantic isolation in traditional KG reasoning, by jointly modeling structural and semantic invariance patterns in graph representations. To demonstrate the effectiveness, we evaluate the inductive reasoning capability of SCR using 38 diverse graph datasets, covering node-level, link-level, and graph-level tasks across multiple domains. Our results show substantial performance gains over existing foundation models and supervised baselines, highlighting the efficacy and adaptability of our approach.

Figures (4)

• Figure 1: The proposed framework SCR transforms diverse graph tasks into inductive reasoning on knowledge graphs with semantic features.
• Figure 2: Preliminary results of the baseline ULTRA. "+Bert" denotes using BERT-encoded features as initialization. We also fine-tune the BERT-encoded ULTRA ("+BertTune"), followed by reasoning with features from the other source ("+OtherTune"). Higher MRR is better.
• Figure 3: The ablation study results of SCR variants. "w/o SARG" denotes using the relation graph without semantic neighbors, the latter two have no semantic-injected initialization and global semantic encoding, respectively. "w/o RelDiff" denotes using identical relation embeddings in SCMP.
• Figure 4: Comparison of performance metrics under different hyperparameter settings ($k$ and $\delta$) for semantic neighbor selection. (a) Average MRR results. (b) Hits@10 results. Darker colors indicate higher values.

Theorems & Definitions (1)

• Definition 4.1