Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Non-Homophilic Graph Pre-Training and Prompt Learning

Xingtong Yu, Jie Zhang, Yuan Fang, Renhe Jiang

TL;DR

The paper addresses pre-training and prompt learning for non-homophilic graphs, where neighbor-label relationships are mixed and not uniformly aligned. It analyzes the limitations of homophily-based pre-training and introduces ProNoG, which uses a condition-net to generate node-specific prompts derived from each node's non-homophilic neighborhood patterns, enabling fine-grained downstream adaptation while freezing the pre-trained encoder. The approach is supported by theoretical insights on why non-homophily tasks are preferable in non-homophilic settings and extensive experiments on ten public datasets showing ProNoG outperforms strong baselines, including uniform graph prompts. The work advances efficient, node-wise adaptation for few-shot graph tasks and offers practical guidance for choosing pre-training objectives in heterogeneous graph settings.

Abstract

Graphs are ubiquitous for modeling complex relationships between objects across various fields. Graph neural networks (GNNs) have become a mainstream technique for graph-based applications, but their performance heavily relies on abundant labeled data. To reduce labeling requirement, pre-training and prompt learning has become a popular alternative. However, most existing prompt methods do not differentiate homophilic and heterophilic characteristics of real-world graphs. In particular, many real-world graphs are non-homophilic, not strictly or uniformly homophilic with mixing homophilic and heterophilic patterns, exhibiting varying non-homophilic characteristics across graphs and nodes. In this paper, we propose ProNoG, a novel pre-training and prompt learning framework for such non-homophilic graphs. First, we analyze existing graph pre-training methods, providing theoretical insights into the choice of pre-training tasks. Second, recognizing that each node exhibits unique non-homophilic characteristics, we propose a conditional network to characterize the node-specific patterns in downstream tasks. Finally, we thoroughly evaluate and analyze ProNoG through extensive experiments on ten public datasets.

Non-Homophilic Graph Pre-Training and Prompt Learning

TL;DR

The paper addresses pre-training and prompt learning for non-homophilic graphs, where neighbor-label relationships are mixed and not uniformly aligned. It analyzes the limitations of homophily-based pre-training and introduces ProNoG, which uses a condition-net to generate node-specific prompts derived from each node's non-homophilic neighborhood patterns, enabling fine-grained downstream adaptation while freezing the pre-trained encoder. The approach is supported by theoretical insights on why non-homophily tasks are preferable in non-homophilic settings and extensive experiments on ten public datasets showing ProNoG outperforms strong baselines, including uniform graph prompts. The work advances efficient, node-wise adaptation for few-shot graph tasks and offers practical guidance for choosing pre-training objectives in heterogeneous graph settings.

Abstract

Graphs are ubiquitous for modeling complex relationships between objects across various fields. Graph neural networks (GNNs) have become a mainstream technique for graph-based applications, but their performance heavily relies on abundant labeled data. To reduce labeling requirement, pre-training and prompt learning has become a popular alternative. However, most existing prompt methods do not differentiate homophilic and heterophilic characteristics of real-world graphs. In particular, many real-world graphs are non-homophilic, not strictly or uniformly homophilic with mixing homophilic and heterophilic patterns, exhibiting varying non-homophilic characteristics across graphs and nodes. In this paper, we propose ProNoG, a novel pre-training and prompt learning framework for such non-homophilic graphs. First, we analyze existing graph pre-training methods, providing theoretical insights into the choice of pre-training tasks. Second, recognizing that each node exhibits unique non-homophilic characteristics, we propose a conditional network to characterize the node-specific patterns in downstream tasks. Finally, we thoroughly evaluate and analyze ProNoG through extensive experiments on ten public datasets.
Paper Structure (22 sections, 2 theorems, 9 equations, 5 figures, 7 tables, 1 algorithm)

This paper contains 22 sections, 2 theorems, 9 equations, 5 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Revisiting Graph Pre-training
  5. Theoretical Insights
  6. Non-homophilic Graph Pre-training
  7. Non-homophilic Prompt Learning
  8. Overall Framework
  9. Prompt Generation and Tuning
  10. Algorithm and Complexity Analysis
  11. Experiments
  12. Experimental Setup
  13. Performance Evaluation
  14. Ablation Study
  15. Analysis on Pre-Training Methods
  16. ...and 7 more sections

Key Result

Theorem 1

For a homophily task $T$, adding a homophily sample always results in a smaller loss than adding a non-homophily sample.

Figures (5)

  • Figure 1: Non-homophilic characteristics of graphs.
  • Figure 2: Overall framework of ProNoG.
  • Figure 3: Impacts of different shots on node classification.
  • Figure 4: Results across nodes with varying homophily ratios.
  • Figure 5: Impact of hidden dimension $m$ in the condition-net.

Theorems & Definitions (4)

  • Definition 1: Homophily Task
  • Definition 2: Homophily Sample
  • Theorem 1
  • Theorem 2