Unsupervised Prompting for Graph Neural Networks

TL;DR

This work addresses GNN generalization under covariate shift without any labeled data by introducing Ugpp, a problem setting where a pre-trained GNN remains frozen and a prompting function is learned solely from unlabeled target data. It introduces UGPrompt, which combines consistency-based prompting with pseudo-labeling and two regularizers (diversity and domain adaptation) to align prompted graphs with source knowledge and mitigate distribution shift. Across graph and node classification benchmarks, UGPrompt often outperforms state-of-the-art prompting methods that rely on labeled data, demonstrating strong unsupervised generalization and robustness. The approach enables leveraging large unlabeled datasets and offers a scalable, task-agnostic framework for prompting frozen GNNs in diverse graph domains.

Abstract

Prompt tuning methods for Graph Neural Networks (GNNs) have become popular to address the semantic gap between pre-training and fine-tuning steps. However, existing GNN prompting methods rely on labeled data and involve lightweight fine-tuning for downstream tasks. Meanwhile, in-context learning methods for Large Language Models (LLMs) have shown promising performance with no parameter updating and no or minimal labeled data. Inspired by these approaches, in this work, we first introduce a challenging problem setup to evaluate GNN prompting methods. This setup encourages a prompting function to enhance a pre-trained GNN's generalization to a target dataset under covariate shift without updating the GNN's parameters and with no labeled data. Next, we propose a fully unsupervised prompting method based on consistency regularization through pseudo-labeling. We use two regularization techniques to align the prompted graphs' distribution with the original data and reduce biased predictions. Through extensive experiments under our problem setting, we demonstrate that our unsupervised approach outperforms the state-of-the-art prompting methods that have access to labels.

Figures (9)

• Figure 1: Overview of UGPrompt. A) A non-parametric algorithm generates a weak augmentation $\mathcal{G}_w$ and a strong augmentation $\mathcal{G}_s$ from an unlabeled graph $\mathcal{G}$. B) The learnable prompting function $f$ generates a prompted graph $\mathcal{G}_p$ from $\mathcal{G}_s$. C) The base Gnn with frozen parameters scores $\mathcal{G}_p$ and $\mathcal{G}_w$. A discriminator taking input from the latent representation ($z_a^{\mathcal{G}}$ for $\mathcal{G}_w$ and $z_p^{\mathcal{G}}$ for $\mathcal{G}_p$) of the Gnn's encoder regularizes the model to adapt to the input distribution.
• Figure 2: The effect of regularization objectives on UGPrompt with GCN as the base model.
• Figure 3: Distribution of embeddings generated by the base Gnn encoder on the PubMed dataset under PR distribution shift with GCN as the base Gnn. Embeddings for BaseModel on source and target test data without prompting and embeddings for UGPrompt and GPF-Plus when target test graphs are prompted are presented. This highlights UGPrompt’s ability to mitigate distribution shift without labeled data, producing well-separated representations similar to the source distribution.
• Figure 4: Graph classification under edge homophily distribution shift for GCN as base model.
• Figure 5: Graph classification under edge homophily distribution shift for GAT as base model.