Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Graph Foundation Models: A Transferability Perspective

Yuxiang Wang, Wenqi Fan, Suhang Wang, Yao Ma

TL;DR

This paper addresses the transferability of Graph Foundation Models (GFMs) across diverse graph domains and tasks. It introduces a transferability-centric taxonomy that segregates GFMs into domain-specific and general-purpose categories and frames their analysis around a three-component life cycle: Backbone Model, Knowledge Acquisition, and Knowledge Transfer. The survey systematically examines backbone architectures (GNNs, Transformers, LLMs, and hybrids) and knowledge strategies (subgraph sampling, SVD alignment, PLM-based semantic alignment, graph prompts, meta-learning, and PEFT) to illuminate how transferability is achieved and where challenges remain. It also outlines future directions, including the integration of LLMs with graph models and the need for standardized benchmarking to gauge cross-domain and cross-task performance.

Abstract

In recent years, Graph Foundation Models (GFMs) have gained significant attention for their potential to generalize across diverse graph domains and tasks. Some works focus on Domain-Specific GFMs, which are designed to address a variety of tasks within a specific domain, while others aim to create General-Purpose GFMs that extend the capabilities of domain-specific models to multiple domains. Regardless of the type, transferability is crucial for applying GFMs across different domains and tasks. However, achieving strong transferability is a major challenge due to the structural, feature, and distributional variations in graph data. To date, there has been no systematic research examining and analyzing GFMs from the perspective of transferability. To bridge the gap, we present the first comprehensive taxonomy that categorizes and analyzes existing GFMs through the lens of transferability, structuring GFMs around their application scope (domain-specific vs. general-purpose) and their approaches to knowledge acquisition and transfer. We provide a structured perspective on current progress and identify potential pathways for advancing GFM generalization across diverse graph datasets and tasks. We aims to shed light on the current landscape of GFMs and inspire future research directions in GFM development.

Towards Graph Foundation Models: A Transferability Perspective

TL;DR

This paper addresses the transferability of Graph Foundation Models (GFMs) across diverse graph domains and tasks. It introduces a transferability-centric taxonomy that segregates GFMs into domain-specific and general-purpose categories and frames their analysis around a three-component life cycle: Backbone Model, Knowledge Acquisition, and Knowledge Transfer. The survey systematically examines backbone architectures (GNNs, Transformers, LLMs, and hybrids) and knowledge strategies (subgraph sampling, SVD alignment, PLM-based semantic alignment, graph prompts, meta-learning, and PEFT) to illuminate how transferability is achieved and where challenges remain. It also outlines future directions, including the integration of LLMs with graph models and the need for standardized benchmarking to gauge cross-domain and cross-task performance.

Abstract

In recent years, Graph Foundation Models (GFMs) have gained significant attention for their potential to generalize across diverse graph domains and tasks. Some works focus on Domain-Specific GFMs, which are designed to address a variety of tasks within a specific domain, while others aim to create General-Purpose GFMs that extend the capabilities of domain-specific models to multiple domains. Regardless of the type, transferability is crucial for applying GFMs across different domains and tasks. However, achieving strong transferability is a major challenge due to the structural, feature, and distributional variations in graph data. To date, there has been no systematic research examining and analyzing GFMs from the perspective of transferability. To bridge the gap, we present the first comprehensive taxonomy that categorizes and analyzes existing GFMs through the lens of transferability, structuring GFMs around their application scope (domain-specific vs. general-purpose) and their approaches to knowledge acquisition and transfer. We provide a structured perspective on current progress and identify potential pathways for advancing GFM generalization across diverse graph datasets and tasks. We aims to shed light on the current landscape of GFMs and inspire future research directions in GFM development.

Paper Structure

This paper contains 39 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. A Taxonomy of GFMs
  3. Domain-Specific vs. General-Purpose GFMs
  4. Transferability: Core Requirement for GFMs
  5. Unified Taxonomy of GFMs
  6. Overall Framework of GFMs
  7. Backbone Model
  8. Common Graph Learning Architectures
  9. GNN-based Models.
  10. Transformer-based Models.
  11. LLM-based Models.
  12. Backbone Models of Existing GFMs
  13. GNN-based models.
  14. Transformer-based models.
  15. LLM-based models.
  16. ...and 24 more sections

Figures (1)

  • Figure 1: The application scope of GFMs. Domain-specific GFMs are designed to handle multiple graph tasks or intra-domain scenarios within a specific domain. General-purpose GFMs go beyond a single domain and extend the capabilities of domain-specific models to multiple domains.