One Prompt Fits All: Universal Graph Adaptation for Pretrained Models

TL;DR

This work tackles the adaptation gap in Graph Prompt Learning (GPL) by identifying two core issues: a lack of unified understanding of how prompts interact with pretrained graph models and limited cross-domain adaptability. It argues that representation-level prompts largely amount to simple linear probes, and thus the research should focus on unleashing pretrained model capabilities via input-level prompts. The authors introduce UniPrompt, a universal GPL that learns a topological prompt graph through a learnable $k$NN-based adjacency while preserving the original graph and training a lightweight classifier, with a bootstrapped integration to prevent collapse. Extensive experiments on homophilic and heterophilic graphs, in-domain and cross-domain, demonstrate that UniPrompt consistently outperforms state-of-the-art GPL baselines with strong robustness and scalability across pretrained models, supporting its potential to advance Graph Foundation Models.

Abstract

Graph Prompt Learning (GPL) has emerged as a promising paradigm that bridges graph pretraining models and downstream scenarios, mitigating label dependency and the misalignment between upstream pretraining and downstream tasks. Although existing GPL studies explore various prompt strategies, their effectiveness and underlying principles remain unclear. We identify two critical limitations: (1) Lack of consensus on underlying mechanisms: Despite current GPLs have advanced the field, there is no consensus on how prompts interact with pretrained models, as different strategies intervene at varying spaces within the model, i.e., input-level, layer-wise, and representation-level prompts. (2) Limited scenario adaptability: Most methods fail to generalize across diverse downstream scenarios, especially under data distribution shifts (e.g., homophilic-to-heterophilic graphs). To address these issues, we theoretically analyze existing GPL approaches and reveal that representation-level prompts essentially function as fine-tuning a simple downstream classifier, proposing that graph prompt learning should focus on unleashing the capability of pretrained models, and the classifier should adapt to downstream scenarios. Based on our findings, we propose UniPrompt, a novel GPL method that adapts any pretrained models, unleashing the capability of pretrained models while preserving the input graph. Extensive experiments demonstrate that our method can effectively integrate with various pretrained models and achieve strong performance across in-domain and cross-domain scenarios.

Figures (4)

• Figure 1: Three different graph prompting mechanisms: input-level prompt (left), layer-wise prompt (middle), and representation-level prompt (right).
• Figure 2: Comparison of fitted normalized loss curves and 1-shot performance across three pretrained models on the Cora dataset. Fine-tune(C) denotes linear probe, while GPPT and GraphPrompt are two GPL methods. Shading indicates the standard deviation of the sliding window.
• Figure 3: In-domain node classification experiments over nine datasets under different shot settings using DGI as the pretrained model.
• Figure 4: Hyperparameter analysis of $\tau$ and $k$ for 1-shot node classification with DGI pretrained.

Theorems & Definitions (4)

• Definition 4.1
• Theorem 4.1: Parameter Objective Equivalence
• Proposition 4.1: Function Space Equivalence
• Proposition 4.2: Optimization Objective Equivalence