Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Structure Prompt Learning: A Novel Methodology to Improve Performance of Graph Neural Networks

Zhenhua Huang, Kunhao Li, Shaojie Wang, Zhaohong Jia, Wentao Zhu, Sharad Mehrotra

TL;DR

This work addresses the limitation that graph neural networks often neglect intrinsic graph structure during task-driven training, leading to suboptimal representations. It introduces Graph Structure Prompt Learning (GPL), a training-time framework that adds task-independent losses, specifically first-order ($\mathcal{L}_{1st}$) and second-order ($\mathcal{L}_{2nd}$) graph-structure prompts, to guide GNNs toward richer structural understanding without requiring additional data or pretraining. GPL is demonstrated to improve node classification, graph classification, and edge prediction across eleven real-world datasets, achieving notable gains and sometimes new state-of-the-art performance, while mitigating over-smoothing. The approach is model-agnostic and practical, offering a new direction for enhancing GNNs with structural prompts and broad potential impact across domains that rely on graph representations.

Abstract

Graph neural networks (GNNs) are widely applied in graph data modeling. However, existing GNNs are often trained in a task-driven manner that fails to fully capture the intrinsic nature of the graph structure, resulting in sub-optimal node and graph representations. To address this limitation, we propose a novel Graph structure Prompt Learning method (GPL) to enhance the training of GNNs, which is inspired by prompt mechanisms in natural language processing. GPL employs task-independent graph structure losses to encourage GNNs to learn intrinsic graph characteristics while simultaneously solving downstream tasks, producing higher-quality node and graph representations. In extensive experiments on eleven real-world datasets, after being trained by GPL, GNNs significantly outperform their original performance on node classification, graph classification, and edge prediction tasks (up to 10.28%, 16.5%, and 24.15%, respectively). By allowing GNNs to capture the inherent structural prompts of graphs in GPL, they can alleviate the issue of over-smooth and achieve new state-of-the-art performances, which introduces a novel and effective direction for GNN research with potential applications in various domains.

Graph Structure Prompt Learning: A Novel Methodology to Improve Performance of Graph Neural Networks

TL;DR

This work addresses the limitation that graph neural networks often neglect intrinsic graph structure during task-driven training, leading to suboptimal representations. It introduces Graph Structure Prompt Learning (GPL), a training-time framework that adds task-independent losses, specifically first-order () and second-order () graph-structure prompts, to guide GNNs toward richer structural understanding without requiring additional data or pretraining. GPL is demonstrated to improve node classification, graph classification, and edge prediction across eleven real-world datasets, achieving notable gains and sometimes new state-of-the-art performance, while mitigating over-smoothing. The approach is model-agnostic and practical, offering a new direction for enhancing GNNs with structural prompts and broad potential impact across domains that rely on graph representations.

Abstract

Graph neural networks (GNNs) are widely applied in graph data modeling. However, existing GNNs are often trained in a task-driven manner that fails to fully capture the intrinsic nature of the graph structure, resulting in sub-optimal node and graph representations. To address this limitation, we propose a novel Graph structure Prompt Learning method (GPL) to enhance the training of GNNs, which is inspired by prompt mechanisms in natural language processing. GPL employs task-independent graph structure losses to encourage GNNs to learn intrinsic graph characteristics while simultaneously solving downstream tasks, producing higher-quality node and graph representations. In extensive experiments on eleven real-world datasets, after being trained by GPL, GNNs significantly outperform their original performance on node classification, graph classification, and edge prediction tasks (up to 10.28%, 16.5%, and 24.15%, respectively). By allowing GNNs to capture the inherent structural prompts of graphs in GPL, they can alleviate the issue of over-smooth and achieve new state-of-the-art performances, which introduces a novel and effective direction for GNN research with potential applications in various domains.
Paper Structure (23 sections, 16 equations, 7 figures, 6 tables)

This paper contains 23 sections, 16 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Proposed Method
  5. Problem Statements
  6. Graph Structure Prompt Learning
  7. Node Classification.
  8. Graph Classification.
  9. Experimental Analysis
  10. Datasets
  11. Baselines
  12. Model Training
  13. Node Classification
  14. Graph Classification
  15. Edge Prediction
  16. ...and 8 more sections

Figures (7)

  • Figure 1: A running example of graph structure prompt learning. The color and size represent nodes' labels and degrees, respectively. (a) The example graph. (b) Nodes clustered by class labels. (c) Nodes clustered by degrees. (d) Nodes clustered by neighboring patterns (degree distributions of neighboring nodes in four-dimension). (e) Structure prompts.
  • Figure 2: Using node features learned by GCN gcn2017 on the Pubmed dataset sen2008 to predict (a) nodes' degree and (b) the sum degree of nodes' neighbors.
  • Figure 3: How learning rate impacts the performance of GCN and GCN+GPL.
  • Figure 4: Visualization of nodes' representation after training on Cora.
  • Figure 5: Visualization of graphs' representation after training on NCI1.
  • ...and 2 more figures