ProG: A Graph Prompt Learning Benchmark

TL;DR

This paper tackles inefficiencies and negative transfer in graph pre-training by proposing graph prompt learning as a data-centric alternative. It introduces ProG, the first comprehensive benchmark for graph prompts, unifying methods into prompts as graphs and prompts as tokens, and evaluating 6 pre-training strategies and 5 prompting techniques across 15 datasets. The authors also release an open-source ProG library that standardizes backbones, pre-training, prompting, and evaluation to enable fair comparisons. Empirically, graph prompts generally outperform supervised and traditional pre-training baselines, demonstrate strong few-shot transfer, mitigate negative transfer, and offer favorable efficiency-flexibility trade-offs, signaling a promising direction for scalable, adaptable graph intelligence.

Abstract

Artificial general intelligence on graphs has shown significant advancements across various applications, yet the traditional 'Pre-train & Fine-tune' paradigm faces inefficiencies and negative transfer issues, particularly in complex and few-shot settings. Graph prompt learning emerges as a promising alternative, leveraging lightweight prompts to manipulate data and fill the task gap by reformulating downstream tasks to the pretext. However, several critical challenges still remain: how to unify diverse graph prompt models, how to evaluate the quality of graph prompts, and to improve their usability for practical comparisons and selection. In response to these challenges, we introduce the first comprehensive benchmark for graph prompt learning. Our benchmark integrates SIX pre-training methods and FIVE state-of-the-art graph prompt techniques, evaluated across FIFTEEN diverse datasets to assess performance, flexibility, and efficiency. We also present 'ProG', an easy-to-use open-source library that streamlines the execution of various graph prompt models, facilitating objective evaluations. Additionally, we propose a unified framework that categorizes existing graph prompt methods into two main approaches: prompts as graphs and prompts as tokens. This framework enhances the applicability and comparison of graph prompt techniques. The code is available at: https://github.com/sheldonresearch/ProG.

Figures (9)

• Figure 1: An overview of our benchmark.
• Figure 2: An overview of ProG
• Figure 3: Head map of GPF-plus and All-in-one on node-level and graph-level tasks across various pre-training methods and datasets. The green interval shows that the graph prompt learning method performs worse than 'Pre-train & Fine-tune'. The yellow-to-red interval shows that the graph prompt learning method performs better. An upward arrow indicates that the 'Pre-train & Fine-tune' method experiences negative transfer, while the graph prompt learning method can alleviate it significantly.
• Figure 4: Analysis of accuracy, training time and tunable parameters of various graph prompt methods. Note that the gray area enclosed by the dashed line represents the scale of the tunable parameters.
• Figure 5: The curve of the shot number from 1 to 10 and the accuracy (%) on Cora and ENZYMES.