Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DAGPrompT: Pushing the Limits of Graph Prompting with a Distribution-aware Graph Prompt Tuning Approach

Qin Chen, Liang Wang, Bo Zheng, Guojie Song

TL;DR

The paper tackles the challenge of bridging the objective gap between pre training and downstream tasks in GNNs, especially for heterophily graphs, by introducing DAGPrompT. It combines Graph Low-Rank Adaptation (GLoRA) to fine tune the encoder with low complexity and a Hop-specific Graph Prompting module to account for varying hop distributions, reformulating downstream tasks into a link prediction framework. Empirical results across 10 datasets and 14 baselines show DAGPrompT achieving state-of-the-art performance on node and graph classification, with notable gains under heterophily and in few-shot settings, while maintaining efficiency. The theoretical analysis supports the use of low-rank adaptation for improved generalization in low data regimes, and transferability experiments demonstrate robust cross-domain performance, underscoring practical impact for scalable graph learning in diverse distributions.

Abstract

The pre-train then fine-tune approach has advanced GNNs by enabling general knowledge capture without task-specific labels. However, an objective gap between pre-training and downstream tasks limits its effectiveness. Recent graph prompting methods aim to close this gap through task reformulations and learnable prompts. Despite this, they struggle with complex graphs like heterophily graphs. Freezing the GNN encoder can reduce the impact of prompting, while simple prompts fail to handle diverse hop-level distributions. This paper identifies two key challenges in adapting graph prompting methods for complex graphs: (1) adapting the model to new distributions in downstream tasks to mitigate pre-training and fine-tuning discrepancies from heterophily and (2) customizing prompts for hop-specific node requirements. To overcome these challenges, we propose Distribution-aware Graph Prompt Tuning (DAGPrompT), which integrates a GLoRA module for optimizing the GNN encoder's projection matrix and message-passing schema through low-rank adaptation. DAGPrompT also incorporates hop-specific prompts accounting for varying graph structures and distributions among hops. Evaluations on 10 datasets and 14 baselines demonstrate that DAGPrompT improves accuracy by up to 4.79 in node and graph classification tasks, setting a new state-of-the-art while preserving efficiency. Codes are available at GitHub.

DAGPrompT: Pushing the Limits of Graph Prompting with a Distribution-aware Graph Prompt Tuning Approach

TL;DR

The paper tackles the challenge of bridging the objective gap between pre training and downstream tasks in GNNs, especially for heterophily graphs, by introducing DAGPrompT. It combines Graph Low-Rank Adaptation (GLoRA) to fine tune the encoder with low complexity and a Hop-specific Graph Prompting module to account for varying hop distributions, reformulating downstream tasks into a link prediction framework. Empirical results across 10 datasets and 14 baselines show DAGPrompT achieving state-of-the-art performance on node and graph classification, with notable gains under heterophily and in few-shot settings, while maintaining efficiency. The theoretical analysis supports the use of low-rank adaptation for improved generalization in low data regimes, and transferability experiments demonstrate robust cross-domain performance, underscoring practical impact for scalable graph learning in diverse distributions.

Abstract

The pre-train then fine-tune approach has advanced GNNs by enabling general knowledge capture without task-specific labels. However, an objective gap between pre-training and downstream tasks limits its effectiveness. Recent graph prompting methods aim to close this gap through task reformulations and learnable prompts. Despite this, they struggle with complex graphs like heterophily graphs. Freezing the GNN encoder can reduce the impact of prompting, while simple prompts fail to handle diverse hop-level distributions. This paper identifies two key challenges in adapting graph prompting methods for complex graphs: (1) adapting the model to new distributions in downstream tasks to mitigate pre-training and fine-tuning discrepancies from heterophily and (2) customizing prompts for hop-specific node requirements. To overcome these challenges, we propose Distribution-aware Graph Prompt Tuning (DAGPrompT), which integrates a GLoRA module for optimizing the GNN encoder's projection matrix and message-passing schema through low-rank adaptation. DAGPrompT also incorporates hop-specific prompts accounting for varying graph structures and distributions among hops. Evaluations on 10 datasets and 14 baselines demonstrate that DAGPrompT improves accuracy by up to 4.79 in node and graph classification tasks, setting a new state-of-the-art while preserving efficiency. Codes are available at GitHub.
Paper Structure (37 sections, 1 theorem, 15 equations, 9 figures, 10 tables, 2 algorithms)

This paper contains 37 sections, 1 theorem, 15 equations, 9 figures, 10 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Notations
  4. Graph Homophily Measurements
  5. Graph Neural Networks (GNNs)
  6. Fine-tuning Pre-trained GNNs
  7. Prompting Pre-trained GNNs
  8. Method
  9. Label-free Pre-Training
  10. Distribution-aware Graph Prompt Tuning
  11. Tuning with Graph Low-Rank Adaptation
  12. Hop-specific Graph Prompting
  13. Unification of Downstream Tasks
  14. Theoretical Analysis
  15. Experiments
  16. ...and 22 more sections

Key Result

Theorem 1

Let $\mathcal{H}$ be a hypothesis class, and $\mathcal{D} = \{(x_i, y_i)\}$ be a dataset of $m$ i.i.d. samples. Suppose the loss function $\ell(h(x), y)$ is bounded by $0 \leq \ell(h(x), y) \leq B$. Then, with probability at least $1 - \delta$, for all $h \in \mathcal{H}$, we have: where $L(h)$ is the true risk, $\hat{L}_{\mathcal{D}}(h)$ is the empirical risk, and $\mathcal{R}_{\mathcal{D}}(\mat

Figures (9)

  • Figure 1: Heterophily diminishes the effectiveness of prompting techniques that freeze the GNN encoder, resulting in indistinguishable node embeddings.
  • Figure 2: Conventional graph prompting techniques are less effective or even detrimental on heterophily datasets. The homophily ratio zhu2020beyond is indicated in the brackets, with a lower ratio representing stronger heterophily.
  • Figure 3: The framework of Distribution-aware Graph Prompt Tuning.
  • Figure 4: Impact of data heterophily on Syn-Chameleon.
  • Figure 5: Impact of shots on Texas and Chameleon.
  • ...and 4 more figures

Theorems & Definitions (1)

  • Theorem 1