Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploiting Text Semantics for Few and Zero Shot Node Classification on Text-attributed Graph

Yuxiang Wang, Xiao Yan, Shiyu Jin, Quanqing Xu, Chuang Hu, Yuanyuan Zhu, Bo Du, Jia Wu, Jiawei Jiang

TL;DR

The paper tackles the challenge of few- and zero-shot node classification on text-attributed graphs by introducing Text Semantics Augmentation (TSA), which enriches supervision with semantic relationships in text. TSA combines a graph encoder, a text encoder, and a dedicated negative-text encoder, optimizing with three losses: $\mathcal{L}_{CL}$ (contrastive), $\mathcal{L}_{PSM}$ (positive semantics matching), and $\mathcal{L}_{NSC}$ (negative semantics contrast). It introduces a scalable text bank and a learnable negative prompt to mine diverse text semantics and enforce opposite semantics, improving robustness and accuracy in both few- and zero-shot settings; a probability-average inference strategy further leverages positive and negative cues. Empirically, TSA outperforms 13 baselines on five datasets, with average improvements of 4.6% (accuracy) and 6.9% (F1) in few-shot, and 8.8% (accuracy) and 9.3% (F1) in zero-shot, demonstrating the practical value of exploiting text semantics for TAG classification. The work advances the state of the art in TAGs by integrating semantic-rich text supervision into end-to-end training and inference, offering a scalable path to reliable classification with limited labeled data.

Abstract

Text-attributed graph (TAG) provides a text description for each graph node, and few- and zero-shot node classification on TAGs have many applications in fields such as academia and social networks. Existing work utilizes various graph-based augmentation techniques to train the node and text embeddings, while text-based augmentations are largely unexplored. In this paper, we propose Text Semantics Augmentation (TSA) to improve accuracy by introducing more text semantic supervision signals. Specifically, we design two augmentation techniques, i.e., positive semantics matching and negative semantics contrast, to provide more reference texts for each graph node or text description. Positive semantic matching retrieves texts with similar embeddings to match with a graph node. Negative semantic contrast adds a negative prompt to construct a text description with the opposite semantics, which is contrasted with the original node and text. We evaluate TSA on 5 datasets and compare with 13 state-of-the-art baselines. The results show that TSA consistently outperforms all baselines, and its accuracy improvements over the best-performing baseline are usually over 5%.

Exploiting Text Semantics for Few and Zero Shot Node Classification on Text-attributed Graph

TL;DR

The paper tackles the challenge of few- and zero-shot node classification on text-attributed graphs by introducing Text Semantics Augmentation (TSA), which enriches supervision with semantic relationships in text. TSA combines a graph encoder, a text encoder, and a dedicated negative-text encoder, optimizing with three losses: (contrastive), (positive semantics matching), and (negative semantics contrast). It introduces a scalable text bank and a learnable negative prompt to mine diverse text semantics and enforce opposite semantics, improving robustness and accuracy in both few- and zero-shot settings; a probability-average inference strategy further leverages positive and negative cues. Empirically, TSA outperforms 13 baselines on five datasets, with average improvements of 4.6% (accuracy) and 6.9% (F1) in few-shot, and 8.8% (accuracy) and 9.3% (F1) in zero-shot, demonstrating the practical value of exploiting text semantics for TAG classification. The work advances the state of the art in TAGs by integrating semantic-rich text supervision into end-to-end training and inference, offering a scalable path to reliable classification with limited labeled data.

Abstract

Text-attributed graph (TAG) provides a text description for each graph node, and few- and zero-shot node classification on TAGs have many applications in fields such as academia and social networks. Existing work utilizes various graph-based augmentation techniques to train the node and text embeddings, while text-based augmentations are largely unexplored. In this paper, we propose Text Semantics Augmentation (TSA) to improve accuracy by introducing more text semantic supervision signals. Specifically, we design two augmentation techniques, i.e., positive semantics matching and negative semantics contrast, to provide more reference texts for each graph node or text description. Positive semantic matching retrieves texts with similar embeddings to match with a graph node. Negative semantic contrast adds a negative prompt to construct a text description with the opposite semantics, which is contrasted with the original node and text. We evaluate TSA on 5 datasets and compare with 13 state-of-the-art baselines. The results show that TSA consistently outperforms all baselines, and its accuracy improvements over the best-performing baseline are usually over 5%.
Paper Structure (17 sections, 1 theorem, 15 equations, 6 figures, 4 tables)

This paper contains 17 sections, 1 theorem, 15 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. The TSA Framework
  4. Overview
  5. Text Semantics Augmentation
  6. Prompt Tuning and Inference
  7. Experimental Evaluation
  8. Experiment Settings
  9. Main Results (RQ1)
  10. Micro Experiments
  11. Related Work
  12. Conclusion
  13. Proof of Theorem 1
  14. Dataset Statistics
  15. Experimental Result and Analysis
  16. ...and 2 more sections

Key Result

Theorem 1

Given the learnable negative prompt $\mathbf{h}$ and hand-crafted negative prompt $\mathbf{X}^{neg}$, the information entropy of the learnable negative prompt $H(\mathbf{h})$ is related to the lower bound of the information entropy of the hand-crafted negative prompt ${\tt LowerBound}(H(\mathbf{X}^{

Figures (6)

  • Figure 1: The contrastive loss of G2P2 (a) and two semantic augmentation techniques proposed by TSA (b,c). The node-text pair of G2P2 is specified by data as each node has a text description, and TSA mines more semantic information for nodes and texts.
  • Figure 2: The overview of TSA.
  • Figure 3: The illustration of probability-average.
  • Figure 4: The time cost comparison of pre-training and prompting for G2P2 and TSA.
  • Figure 5: The comparison of the number of similar texts and the capacity of text bank for TSA on M.I. anf Industrial.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof