GraphBridge: Towards Arbitrary Transfer Learning in GNNs

TL;DR

GraphBridge tackles the challenge of transferring knowledge across heterogeneous graph tasks and domains by introducing a two-stage pre-training and tuning framework. It augments frozen GNN backbones with an Input Bridge, Output Bridge, and a trainable Graph Side-tuning path, supported by GSST and GMST to mitigate negative transfer and enable outputs of arbitrary dimensions. Across 16 datasets and four transfer scenarios (Graph2Graph, Node2Node, Graph2Node, Graph2PtCld), GraphBridge demonstrates task- and domain-agnostic transfer with a tunable parameter budget as low as $5\% \sim 20\%$, highlighting both effectiveness and resource efficiency. The work also provides a Task Pyramid to structure transfer challenges and confirms the framework's capacity to transfer knowledge from graph domains to graph-like data, advancing practical universal graph transfer learning.

Abstract

Graph neural networks (GNNs) are conventionally trained on a per-domain, per-task basis. It creates a significant barrier in transferring the acquired knowledge to different, heterogeneous data setups. This paper introduces GraphBridge, a novel framework to enable knowledge transfer across disparate tasks and domains in GNNs, circumventing the need for modifications to task configurations or graph structures. Specifically, GraphBridge allows for the augmentation of any pre-trained GNN with prediction heads and a bridging network that connects the input to the output layer. This architecture not only preserves the intrinsic knowledge of the original model but also supports outputs of arbitrary dimensions. To mitigate the negative transfer problem, GraphBridge merges the source model with a concurrently trained model, thereby reducing the source bias when applied to the target domain. Our method is thoroughly evaluated across diverse transfer learning scenarios, including Graph2Graph, Node2Node, Graph2Node, and graph2point-cloud. Empirical validation, conducted over 16 datasets representative of these scenarios, confirms the framework's capacity for task- and domain-agnostic transfer learning within graph-like data, marking a significant advancement in the field of GNNs. Code is available at https://github.com/jujulili888/GraphBridge.

Figures (11)

• Figure 1: Task Pyramid & Core Methodology. Left: Graph side-tuning metods proposed to solve the different difficulty-level of the tasks; Right: Graph transfer learning tasks with different levels of difficulty defined in our work.
• Figure 2: GraphBridge Framework.Left: End-to-end GraphBridge framework with 2 stage; Right: Architecture of Graph-Merge-Side-Tuning architecture for addressing negative transfer.
• Figure 3: Adjustable parameter sizes in different tuning algorithms across distinct backbones. We conduct statistics on five-layer backbones.
• Figure 4: Training speed-up of different tuning methods compared to Scratch Training. We selected the transfer experiment on Cora dataset in the challenging scenario as a representative.
• Figure 5: All versions of Graph Side-tuning architectures.(a) G-Block Side-tune: The Simplest version of the Graph Side-tuning architecture, which has separated base and side networks; (b) G-Assemb Side-tune: G-Block Side-tune with a backup model designed in base model for negative transfer alleviation; (c) G-Scaff Side-tune: Graph Side-tune architecture with layer-wise fusion between base model and side network; (d) G-Merge Side-tune: G-Scaff Side-tune with a backup model designed in base for negative transfer alleviation.