Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GP2F: Cross-Domain Graph Prompting with Adaptive Fusion of Pre-trained Graph Neural Networks

Dongxiao He, Wenxuan Sun, Yongqi Huang, Jitao Zhao, Di Jin

TL;DR

GP2F is proposed, a dual-branch GPL method that explicitly instantiates the two extremes: a frozen branch that retains pre-trained knowledge, and an adapted branch with lightweight adapters for task-specific adaptation that outperforms existing methods on cross-domain few-shot node and graph classification.

Abstract

Graph Prompt Learning (GPL) has recently emerged as a promising paradigm for downstream adaptation of pre-trained graph models, mitigating the misalignment between pre-training objectives and downstream tasks. Recently, the focus of GPL has shifted from in-domain to cross-domain scenarios, which is closer to the real world applications, where the pre-training source and downstream target often differ substantially in data distribution. However, why GPLs remain effective under such domain shifts is still unexplored. Empirically, we observe that representative GPL methods are competitive with two simple baselines in cross-domain settings: full fine-tuning (FT) and linear probing (LP), motivating us to explore a deeper understanding of the prompting mechanism. We provide a theoretical analysis demonstrating that jointly leveraging these two complementary branches yields a smaller estimation error than using either branch alone, formally proving that cross-domain GPL benefits from the integration between pre-trained knowledge and task-specific adaptation. Based on this insight, we propose GP2F, a dual-branch GPL method that explicitly instantiates the two extremes: (1) a frozen branch that retains pre-trained knowledge, and (2) an adapted branch with lightweight adapters for task-specific adaptation. We then perform adaptive fusion under topology constraints via a contrastive loss and a topology-consistent loss. Extensive experiments on cross-domain few-shot node and graph classification demonstrate that our method outperforms existing methods.

GP2F: Cross-Domain Graph Prompting with Adaptive Fusion of Pre-trained Graph Neural Networks

TL;DR

GP2F is proposed, a dual-branch GPL method that explicitly instantiates the two extremes: a frozen branch that retains pre-trained knowledge, and an adapted branch with lightweight adapters for task-specific adaptation that outperforms existing methods on cross-domain few-shot node and graph classification.

Abstract

Graph Prompt Learning (GPL) has recently emerged as a promising paradigm for downstream adaptation of pre-trained graph models, mitigating the misalignment between pre-training objectives and downstream tasks. Recently, the focus of GPL has shifted from in-domain to cross-domain scenarios, which is closer to the real world applications, where the pre-training source and downstream target often differ substantially in data distribution. However, why GPLs remain effective under such domain shifts is still unexplored. Empirically, we observe that representative GPL methods are competitive with two simple baselines in cross-domain settings: full fine-tuning (FT) and linear probing (LP), motivating us to explore a deeper understanding of the prompting mechanism. We provide a theoretical analysis demonstrating that jointly leveraging these two complementary branches yields a smaller estimation error than using either branch alone, formally proving that cross-domain GPL benefits from the integration between pre-trained knowledge and task-specific adaptation. Based on this insight, we propose GP2F, a dual-branch GPL method that explicitly instantiates the two extremes: (1) a frozen branch that retains pre-trained knowledge, and (2) an adapted branch with lightweight adapters for task-specific adaptation. We then perform adaptive fusion under topology constraints via a contrastive loss and a topology-consistent loss. Extensive experiments on cross-domain few-shot node and graph classification demonstrate that our method outperforms existing methods.
Paper Structure (31 sections, 3 theorems, 28 equations, 8 figures, 9 tables, 1 algorithm)

This paper contains 31 sections, 3 theorems, 28 equations, 8 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Theoretical analysis
  4. Method
  5. Dual-Branch Encoding
  6. Residual Adapter with Structural Contrastive Loss
  7. Topology-Consistent Fusion Loss
  8. Optimization Objective
  9. Experiments
  10. Experimental setup
  11. Performance of Cross-Domain Classification (RQ1)
  12. Robustness Across Pre-training Strategies (RQ2)
  13. Comparison with Graph Foundation Models (RQ3)
  14. Scalability on Large-Scale Datasets (RQ4)
  15. Model Analysis (RQ5)
  16. ...and 16 more sections

Key Result

Lemma 3.3

Under Assumptions ass:error-model, define $\sigma_g^2 := \mathbb{E}[\|\boldsymbol{\epsilon}_i^g\|^2]$, $\sigma_a^2 := \mathbb{E}[\|\boldsymbol{\epsilon}_i^a\|^2]$, and $\rho := \mathbb{E}[\langle \boldsymbol{\epsilon}_i^g, \boldsymbol{\epsilon}_i^a\rangle]$. Then $\mathrm{MSE}(\lambda)$ is a strictl where $A=\sigma_g^2+\sigma_a^2-2\rho >0$, $B=2(\rho-\sigma_a^2)$ and $C=\sigma_a^2$. The $\lambda^\

Figures (8)

  • Figure 1: In cross-domain 1-shot scenario, where the GNN is pre-trained on Cora using GRACE, competitive performance is observed between representative GPL methods and two strong baselines, full Fine-Tuning (FT) and Linear Probing (LP).
  • Figure 2: Overall framework of the proposed GP2F. Given a graph in target domain $\mathcal{D}_\mathrm{T}$ and a encoder pre-trained on source domain $\mathcal{D}_\mathrm{S}$, ${Proj}(\cdot)$ is a MLP used for dimension alignment. The frozen branch consists of a pre-trained GNN to preserve universal knowledge, while the adapted branch uses learnable adapters $\mathcal{A}$ for downstream adaptation. Additionally, a contrastive loss $\mathcal{L}_{\mathrm{ctr}}$ is used to align the two branches, and a BCE loss constrains the fusion. $\mathbf{H}_{\mathrm{mix}}$ is used for downstream prediction.
  • Figure 3: Accuracy of 3-shot and 5-shot cross-domain node classification experiments using GRACE for pre-training.
  • Figure 4: Analysis of key components in GP2F via 1-shot node classification pre-trained with GRACE.
  • Figure 5: Hyperparameter analysis of $r$ for 1-shot node classification with GRACE pre-trained.
  • ...and 3 more figures

Theorems & Definitions (6)

  • Lemma 3.3: Quadratic form of the MSE
  • Theorem 3.4: Strict MSE improvement over either branch
  • proof
  • proof
  • Corollary 1.2: Margin-based misclassification bound
  • proof