All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining

TL;DR

Cross-domain graph pretraining often suffers negative transfer when aggregating diverse graph datasets, hindering few-shot transfer. The authors propose GCOPE, a simple yet effective framework that introduces learnable coordinators to connect multiple graphs, align their features via a projection, and jointly pretrain with a contrastive objective plus a feature-reconstruction loss, forming a unified adjacency $\tilde{A}$ and enabling cross-domain knowledge transfer. Empirically, GCOPE yields strong positive transfer across both homophilic and heterophilic graphs, with analyses showing the crucial role of inter-coordinator edges and a balanced reconstruction term; it also demonstrates compatibility with graph prompting for downstream tasks. This work advances graph foundational models by enabling a unified, multi-domain pretraining paradigm that leverages diverse graph topologies and semantics to improve few-shot generalization and cross-domain robustness.

Abstract

Large Language Models (LLMs) have revolutionized the fields of computer vision (CV) and natural language processing (NLP). One of the most notable advancements of LLMs is that a single model is trained on vast and diverse datasets spanning multiple domains -- a paradigm we term `All in One'. This methodology empowers LLMs with super generalization capabilities, facilitating an encompassing comprehension of varied data distributions. Leveraging these capabilities, a single LLM demonstrates remarkable versatility across a variety of domains -- a paradigm we term `One for All'. However, applying this idea to the graph field remains a formidable challenge, with cross-domain pretraining often resulting in negative transfer. This issue is particularly important in few-shot learning scenarios, where the paucity of training data necessitates the incorporation of external knowledge sources. In response to this challenge, we propose a novel approach called Graph COordinators for PrEtraining (GCOPE), that harnesses the underlying commonalities across diverse graph datasets to enhance few-shot learning. Our novel methodology involves a unification framework that amalgamates disparate graph datasets during the pretraining phase to distill and transfer meaningful knowledge to target tasks. Extensive experiments across multiple graph datasets demonstrate the superior efficacy of our approach. By successfully leveraging the synergistic potential of multiple graph datasets for pretraining, our work stands as a pioneering contribution to the realm of graph foundational model.

Figures (3)

• Figure 1: Negative transfer phenomenon in the single-source cross-domain transfer setting. Sources (Pubmed and Photos) are two homophilic datasets. Targets (Wisconsin, Texas, Cornell, Chameleon, and Squirrel) are five heterophilic graphs.
• Figure 2: Overview of our proposed GCOPE method. The left part is our pretraining stage and the right part transferring stage.
• Figure 3: Node classification performance (mean±std) of GCOPE with varying reconstruction loss coefficient ($\lambda$) values on Citeseer under C-way-1-shot setting.