Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Unified Graph Selective Prompt Learning for Graph Neural Networks

Bo Jiang, Hao Wu, Ziyan Zhang, Beibei Wang, Jin Tang

TL;DR

This paper addresses the gap between pre-training and downstream tasks in graph neural networks by introducing Graph Selective Prompt Feature learning (GSPF). GSPF unifies node and edge prompting and adds selectivity through node importance $r_i$ and edge attention to concentrate prompts on informative components, while keeping most pre-trained parameters fixed. Empirical results on large molecular datasets demonstrate that GSPF outperforms full fine-tuning and prior graph prompt methods across multiple pre-training strategies, with strong robustness to data size and lower parameter cost. The approach offers a practical, parameter-efficient path to adapting pre-trained GNNs for graph-level and node-level tasks in chemistry and related domains.

Abstract

In recent years, graph prompt learning/tuning has garnered increasing attention in adapting pre-trained models for graph representation learning. As a kind of universal graph prompt learning method, Graph Prompt Feature (GPF) has achieved remarkable success in adapting pre-trained models for Graph Neural Networks (GNNs). By fixing the parameters of a pre-trained GNN model, the aim of GPF is to modify the input graph data by adding some (learnable) prompt vectors into graph node features to better align with the downstream tasks on the smaller dataset. However, existing GPFs generally suffer from two main limitations. First, GPFs generally focus on node prompt learning which ignore the prompting for graph edges. Second, existing GPFs generally conduct the prompt learning on all nodes equally which fails to capture the importances of different nodes and may perform sensitively w.r.t noisy nodes in aligning with the downstream tasks. To address these issues, in this paper, we propose a new unified Graph Selective Prompt Feature learning (GSPF) for GNN fine-tuning. The proposed GSPF integrates the prompt learning on both graph node and edge together, which thus provides a unified prompt model for the graph data. Moreover, it conducts prompt learning selectively on nodes and edges by concentrating on the important nodes and edges for prompting which thus make our model be more reliable and compact. Experimental results on many benchmark datasets demonstrate the effectiveness and advantages of the proposed GSPF method.

A Unified Graph Selective Prompt Learning for Graph Neural Networks

TL;DR

This paper addresses the gap between pre-training and downstream tasks in graph neural networks by introducing Graph Selective Prompt Feature learning (GSPF). GSPF unifies node and edge prompting and adds selectivity through node importance and edge attention to concentrate prompts on informative components, while keeping most pre-trained parameters fixed. Empirical results on large molecular datasets demonstrate that GSPF outperforms full fine-tuning and prior graph prompt methods across multiple pre-training strategies, with strong robustness to data size and lower parameter cost. The approach offers a practical, parameter-efficient path to adapting pre-trained GNNs for graph-level and node-level tasks in chemistry and related domains.

Abstract

In recent years, graph prompt learning/tuning has garnered increasing attention in adapting pre-trained models for graph representation learning. As a kind of universal graph prompt learning method, Graph Prompt Feature (GPF) has achieved remarkable success in adapting pre-trained models for Graph Neural Networks (GNNs). By fixing the parameters of a pre-trained GNN model, the aim of GPF is to modify the input graph data by adding some (learnable) prompt vectors into graph node features to better align with the downstream tasks on the smaller dataset. However, existing GPFs generally suffer from two main limitations. First, GPFs generally focus on node prompt learning which ignore the prompting for graph edges. Second, existing GPFs generally conduct the prompt learning on all nodes equally which fails to capture the importances of different nodes and may perform sensitively w.r.t noisy nodes in aligning with the downstream tasks. To address these issues, in this paper, we propose a new unified Graph Selective Prompt Feature learning (GSPF) for GNN fine-tuning. The proposed GSPF integrates the prompt learning on both graph node and edge together, which thus provides a unified prompt model for the graph data. Moreover, it conducts prompt learning selectively on nodes and edges by concentrating on the important nodes and edges for prompting which thus make our model be more reliable and compact. Experimental results on many benchmark datasets demonstrate the effectiveness and advantages of the proposed GSPF method.
Paper Structure (20 sections, 10 equations, 4 figures, 3 tables)

This paper contains 20 sections, 10 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Pre-training on Graphs
  4. Prompt Learning on Graphs
  5. The Proposed Method
  6. Preliminaries
  7. Graph Selective Prompt Learning
  8. Selective Node Prompt
  9. Selective Edge Prompt
  10. Experiment
  11. Datasets
  12. Pre-training and Prompting Methods
  13. Experimental Setup
  14. Comparison Results
  15. Model Analyses
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Illustration of our proposed GSPF approach. It integrates node prompt for node feature enrichment and edge prompt for adjacency matrix modification.
  • Figure 2: ROC-AUC curves of AttrMasking 0:hu2020strategies and ContextPred 0:hu2020strategies with FT and our GSPF method on the Tox21 0:wu2018moleculenet dataset.
  • Figure 3: Comparison of test ROC-AUC (%) performances using various training data percentages on SIDER 0:wu2018moleculenet and ToxCast 0:wu2018moleculenet datasets.
  • Figure 4: Comparative ROC-AUC (%) percentages for edge prompt at different layer levels: 'shallow' denotes applying edge prompt only at the first GNN layer, while 'deep' denotes applying edge prompt at each GNN layer.