GraphPrompter: Multi-stage Adaptive Prompt Optimization for Graph In-Context Learning

TL;DR

This work addresses the limitations of existing graph in-context learning methods, which rely on noisy, randomly selected prompts and suffer from domain shift when downstream graphs differ from pretraining data. It introduces GraphPrompter, a multi-stage framework that jointly optimizes prompt generation, selection, and online augmentation to improve in-context learning without updating graph model parameters. Through a Prompt Generator with edge-weight reconstruction, a Prompt Selector with learnable importance plus kNN retrieval, and a Prompt Augmenter using an LFU cache, the approach achieves robust gains across node and edge classification tasks and demonstrates strong generalization to multi-class downstream tasks. The results show consistent, significant improvements over baselines (average >8%), with scalable performance on large-scale graphs and practical implications for graph understanding in cross-domain settings.

Abstract

Graph In-Context Learning, with the ability to adapt pre-trained graph models to novel and diverse downstream graphs without updating any parameters, has gained much attention in the community. The key to graph in-context learning is to perform downstream graphs conditioned on chosen prompt examples. Existing methods randomly select subgraphs or edges as prompts, leading to noisy graph prompts and inferior model performance. Additionally, due to the gap between pre-training and testing graphs, when the number of classes in the testing graphs is much greater than that in the training, the in-context learning ability will also significantly deteriorate. To tackle the aforementioned challenges, we develop a multi-stage adaptive prompt optimization method GraphPrompter, which optimizes the entire process of generating, selecting, and using graph prompts for better in-context learning capabilities. Firstly, Prompt Generator introduces a reconstruction layer to highlight the most informative edges and reduce irrelevant noise for graph prompt construction. Furthermore, in the selection stage, Prompt Selector employs the $k$-nearest neighbors algorithm and pre-trained selection layers to dynamically choose appropriate samples and minimize the influence of irrelevant prompts. Finally, we leverage a Prompt Augmenter with a cache replacement strategy to enhance the generalization capability of the pre-trained model on new datasets. Extensive experiments show that GraphPrompter effectively enhances the in-context learning ability of graph models. On average across all the settings, our approach surpasses the state-of-the-art baselines by over 8%. Our code is released at https://github.com/karin0018/GraphPrompter.

Figures (10)

• Figure 1: Graph In-Context Learning (edge classification as an example) with random prompts selection. The graph prompts are sampled $l$-hop subgraphs (left) and the downstream tasks are reformulated as the edge label predictions between the query data graph and label nodes (right).
• Figure 2: An overview of the GraphPrompter method. Overall the method can be divided into three components: (a) Prompt Generator. We select candidate prompt subgraphs $\mathcal{S}$ and query subgraphs $\mathcal{Q}$ and filter their edges by the pre-training edge weights(represented by varying shades). (b) In Prompt Selector stage, after obtaining embeddings for subgraphs through GNN layers, we computed the probability (Score) of each prompt being selected as the combination of the $k$NN similarity and the importance value obtained from the pre-trained selection layer to adaptively select the top-$k$ suitable prompts. These prompts form the new prompt set $\mathcal{\hat{S}}$. (c) In Prompt Augmenter, we utilize a cache $\mathcal{C}$ to store online test samples and their pseudo labels, which helps enhance the prompt set dynamically. We predict the query using adaptive prompt selection and optionally consider prompts from the cache. All the model parameters are learned in the pretraining phase.
• Figure 3: Ablation study on FB15K-237 and NELL with 3-shots wrt. the number of ways from 5 to 40.
• Figure 4: The comparison of different GNN architectures on FB15K-237 and NELL datasets.
• Figure 5: Analysis of cache size from 1 to 10. FB noted FB15K-237 dataset and NELL means NELL dataset.