Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models

Xingtong Yu, Chang Zhou, Yuan Fang, Xinming Zhang

TL;DR

MDGPT tackles the challenge of training a graph foundation model across diverse, text-free domains by introducing domain tokens to align heterogeneous node features and a dual-prompt framework (unifying and mixing prompts) to adapt to downstream tasks. The method performs multi-domain pre-training with a dimension alignment step and a universal self-supervised task template, followed by lightweight downstream adaptation that freezes the pre-trained encoder. Across six public datasets and both seen/unseen domains, MDGPT substantially outperforms state-of-the-art baselines in one-shot and few-shot settings, while maintaining high parameter efficiency. This approach enables robust cross-domain generalization for graph analytics without relying on textual descriptions, with strong implications for scalable, foundation-model-style graph learning in practice.

Abstract

Given the ubiquity of graph data, it is intriguing to ask: Is it possible to train a graph foundation model on a broad range of graph data across diverse domains? A major hurdle toward this goal lies in the fact that graphs from different domains often exhibit profoundly divergent characteristics. Although there have been some initial efforts in integrating multi-domain graphs for pre-training, they primarily rely on textual descriptions to align the graphs, limiting their application to text-attributed graphs. Moreover, different source domains may conflict or interfere with each other, and their relevance to the target domain can vary significantly. To address these issues, we propose MDGPT, a text free Multi-Domain Graph Pre-Training and adaptation framework designed to exploit multi-domain knowledge for graph learning. First, we propose a set of domain tokens to to align features across source domains for synergistic pre-training. Second, we propose a dual prompts, consisting of a unifying prompt and a mixing prompt, to further adapt the target domain with unified multi-domain knowledge and a tailored mixture of domain-specific knowledge. Finally, we conduct extensive experiments involving six public datasets to evaluate and analyze MDGPT, which outperforms prior art by up to 37.9%.

Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models

TL;DR

MDGPT tackles the challenge of training a graph foundation model across diverse, text-free domains by introducing domain tokens to align heterogeneous node features and a dual-prompt framework (unifying and mixing prompts) to adapt to downstream tasks. The method performs multi-domain pre-training with a dimension alignment step and a universal self-supervised task template, followed by lightweight downstream adaptation that freezes the pre-trained encoder. Across six public datasets and both seen/unseen domains, MDGPT substantially outperforms state-of-the-art baselines in one-shot and few-shot settings, while maintaining high parameter efficiency. This approach enables robust cross-domain generalization for graph analytics without relying on textual descriptions, with strong implications for scalable, foundation-model-style graph learning in practice.

Abstract

Given the ubiquity of graph data, it is intriguing to ask: Is it possible to train a graph foundation model on a broad range of graph data across diverse domains? A major hurdle toward this goal lies in the fact that graphs from different domains often exhibit profoundly divergent characteristics. Although there have been some initial efforts in integrating multi-domain graphs for pre-training, they primarily rely on textual descriptions to align the graphs, limiting their application to text-attributed graphs. Moreover, different source domains may conflict or interfere with each other, and their relevance to the target domain can vary significantly. To address these issues, we propose MDGPT, a text free Multi-Domain Graph Pre-Training and adaptation framework designed to exploit multi-domain knowledge for graph learning. First, we propose a set of domain tokens to to align features across source domains for synergistic pre-training. Second, we propose a dual prompts, consisting of a unifying prompt and a mixing prompt, to further adapt the target domain with unified multi-domain knowledge and a tailored mixture of domain-specific knowledge. Finally, we conduct extensive experiments involving six public datasets to evaluate and analyze MDGPT, which outperforms prior art by up to 37.9%.
Paper Structure (19 sections, 7 equations, 5 figures, 8 tables)

This paper contains 19 sections, 7 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Proposed Approach
  5. Overall framework
  6. Multi-domain pre-training
  7. Downstream adaptation
  8. Experiments
  9. Experimental Setup
  10. Few-shot performance evaluation
  11. Ablation Study
  12. Hyperparameter Analysis
  13. Homophily Sensitivity
  14. Node Classification on Seen Domains
  15. Parameters Efficiency
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Motivation of MDGPT. (a) Different scenarios of pre-training and downstream adaptation. (b) Accuracy of one-shot node classification with DGI on two target domains, namely, Cora and Photo, as more source domains are added to pre-training.
  • Figure 2: Overall framework of MDGPT.
  • Figure 3: Impact of number of shots on node classification on three target domains.
  • Figure 4: Data ablation study with a growing number of source domains.
  • Figure 5: Sensitivity study of $\tilde{d}$, the aligned feature dimension across domains, on three target domains.