Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Edge Prompt Tuning for Graph Neural Networks

Xingbo Fu, Yinhan He, Jundong Li

TL;DR

EdgePrompt addresses the objective gap between self-supervised graph pre-training and downstream tasks by introducing edge-centric prompts that are integrated into message passing. It proposes EdgePrompt and EdgePrompt+—edge-focused prompt designs with a global per-layer prompt and, in EdgePrompt+, anchor prompts plus edge-specific weights—to enhance the structural embedding learned by frozen pre-trained GNNs. The authors provide theoretical analyses showing improved linear separability and universality for graph representations, and validate the approach with extensive experiments across ten datasets and four pre-training strategies, where EdgePrompt+ often achieves state-of-the-art results. The work demonstrates the practicality and effectiveness of leveraging edge information in graph prompt tuning, with competitive efficiency and guidance on the number of anchor prompts.

Abstract

Pre-training powerful Graph Neural Networks (GNNs) with unlabeled graph data in a self-supervised manner has emerged as a prominent technique in recent years. However, inevitable objective gaps often exist between pre-training and downstream tasks. To bridge this gap, graph prompt tuning techniques design and learn graph prompts by manipulating input graphs or reframing downstream tasks as pre-training tasks without fine-tuning the pre-trained GNN models. While recent graph prompt tuning methods have proven effective in adapting pre-trained GNN models for downstream tasks, they overlook the crucial role of edges in graph prompt design, which can significantly affect the quality of graph representations for downstream tasks. In this study, we propose EdgePrompt, a simple yet effective graph prompt tuning method from the perspective of edges. Unlike previous studies that design prompt vectors on node features, EdgePrompt manipulates input graphs by learning additional prompt vectors for edges and incorporates the edge prompts through message passing in the pre-trained GNN models to better embed graph structural information for downstream tasks. Our method is compatible with prevalent GNN architectures pre-trained under various pre-training strategies and is universal for different downstream tasks. We provide comprehensive theoretical analyses of our method regarding its capability of handling node classification and graph classification as downstream tasks. Extensive experiments on ten graph datasets under four pre-training strategies demonstrate the superiority of our proposed method against six baselines. Our code is available at https://github.com/xbfu/EdgePrompt.

Edge Prompt Tuning for Graph Neural Networks

TL;DR

EdgePrompt addresses the objective gap between self-supervised graph pre-training and downstream tasks by introducing edge-centric prompts that are integrated into message passing. It proposes EdgePrompt and EdgePrompt+—edge-focused prompt designs with a global per-layer prompt and, in EdgePrompt+, anchor prompts plus edge-specific weights—to enhance the structural embedding learned by frozen pre-trained GNNs. The authors provide theoretical analyses showing improved linear separability and universality for graph representations, and validate the approach with extensive experiments across ten datasets and four pre-training strategies, where EdgePrompt+ often achieves state-of-the-art results. The work demonstrates the practicality and effectiveness of leveraging edge information in graph prompt tuning, with competitive efficiency and guidance on the number of anchor prompts.

Abstract

Pre-training powerful Graph Neural Networks (GNNs) with unlabeled graph data in a self-supervised manner has emerged as a prominent technique in recent years. However, inevitable objective gaps often exist between pre-training and downstream tasks. To bridge this gap, graph prompt tuning techniques design and learn graph prompts by manipulating input graphs or reframing downstream tasks as pre-training tasks without fine-tuning the pre-trained GNN models. While recent graph prompt tuning methods have proven effective in adapting pre-trained GNN models for downstream tasks, they overlook the crucial role of edges in graph prompt design, which can significantly affect the quality of graph representations for downstream tasks. In this study, we propose EdgePrompt, a simple yet effective graph prompt tuning method from the perspective of edges. Unlike previous studies that design prompt vectors on node features, EdgePrompt manipulates input graphs by learning additional prompt vectors for edges and incorporates the edge prompts through message passing in the pre-trained GNN models to better embed graph structural information for downstream tasks. Our method is compatible with prevalent GNN architectures pre-trained under various pre-training strategies and is universal for different downstream tasks. We provide comprehensive theoretical analyses of our method regarding its capability of handling node classification and graph classification as downstream tasks. Extensive experiments on ten graph datasets under four pre-training strategies demonstrate the superiority of our proposed method against six baselines. Our code is available at https://github.com/xbfu/EdgePrompt.

Paper Structure

This paper contains 26 sections, 2 theorems, 29 equations, 5 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Methodology
  5. Problem Setting
  6. Edge Prompt Design
  7. Analysis of Edge Prompt Tuning for Node Classification
  8. Extension to Graph Classification
  9. Experiments
  10. Experimental Setup
  11. Main Results
  12. Convergence Analysis
  13. Influence of Prompt Numbers
  14. Conclusion
  15. Proof of Theorem \ref{['theorem: edgeprompt+']}
  16. ...and 11 more sections

Key Result

Theorem 1

Given a random graph ${\mathcal{G}} \sim CSBM ({\bm{\mu}}_1, {\bm{\mu}}_2, p, q)$ and a pre-trained GCN model $f$, there always exist a set of $M \geq 2$ anchor prompts ${\mathcal{P}}=\{{\bm{p}}_1, {\bm{p}}_2, \cdots, {\bm{p}}_{M}\}$ and the score vectors ${\bm{b}}_{i, j}$ for each edge $(v_i, v_j)$

Figures (5)

  • Figure 1: Learning prompt vectors on a node may uniformly pass them to its neighboring nodes while learning prompt vectors on edges can result in customized prompt aggregation.
  • Figure 2: Convergence speeds of different methods.
  • Figure 3: Results of EdgePrompt+ with varying numbers of anchor prompts on node classification.
  • Figure 4: Results of EdgePrompt+ with varying numbers of anchor prompts on graph classification.
  • Figure 5: Convergence speeds of different methods.

Theorems & Definitions (5)

  • Theorem 1
  • Theorem 2
  • proof
  • proof
  • proof