Prompt Tuning without Labeled Samples for Zero-Shot Node Classification in Text-Attributed Graphs

TL;DR

This work tackles zero-shot $N$-way node classification on text-attributed graphs (TAGs) with no labeled data. It introduces a three-phase framework that first pre-trains a graph-language model and then trains a Universal Bimodal Conditional Generator (UBCG) to synthesize class-conditioned multi-modal samples, enabling continuous prompt tuning using only class names. Key contributions include a label-free zero-shot prompt tuning mechanism, a universal bimodal generator that can produce both graph and text embeddings, and comprehensive experiments across four TAG datasets showing strong performance and robust ablations. The approach reduces labeling costs and enables scalable generalization to unseen classes in realistic networked data settings.

Abstract

Node classification is a fundamental problem in information retrieval with many real-world applications, such as community detection in social networks, grouping articles published online and product categorization in e-commerce. Zero-shot node classification in text-attributed graphs (TAGs) presents a significant challenge, particularly due to the absence of labeled data. In this paper, we propose a novel Zero-shot Prompt Tuning (ZPT) framework to address this problem by leveraging a Universal Bimodal Conditional Generator (UBCG). Our approach begins with pre-training a graph-language model to capture both the graph structure and the associated textual descriptions of each node. Following this, a conditional generative model is trained to learn the joint distribution of nodes in both graph and text modalities, enabling the generation of synthetic samples for each class based solely on the class name. These synthetic node and text embeddings are subsequently used to perform continuous prompt tuning, facilitating effective node classification in a zero-shot setting. Furthermore, we conduct extensive experiments on multiple benchmark datasets, demonstrating that our framework performs better than existing state-of-the-art baselines. We also provide ablation studies to validate the contribution of the bimodal generator. The code is provided at: https://github.com/Sethup123/ZPT.

Figures (5)

• Figure 1: Illustration of different prompt learning techniques. The frozen symbol indicates the parameters are frozen; the fire symbol indicates that the parameters are tuned.
• Figure 2: Overall framework of our zero-shot prompt tuning (ZPT). (a) and (b) are performed as pre-training steps, and (c) denotes the downstream classification phase.
• Figure 3: Performance of pseudo-labels vs. synthetic samples.
• Figure 4: Sensitivity analysis of model parameters.
• Figure 5: $t$-SNE visualization of real and synthetic samples, as represented by dark and faded markers, respectively.