Unsupervised Episode Generation for Graph Meta-learning

TL;DR

This paper tackles the Few-Shot Node Classification problem under label scarcity by proposing NaQ, an unsupervised episode-generation method for graph meta-learning. NaQ constructs training episodes by random 1-shot support sets and query sets built from feature- or diffusion-based similar neighbors, enabling downstream-task-aware meta-learning without labeled data and making full use of all graph nodes. Empirically, NaQ-Feat and NaQ-Diff outperform strong supervised baselines and several GCL methods across multiple datasets, including large-scale ogbn-arxiv and ogbn-products, while showing robustness to class imbalance and strong model-agnostic applicability. Limitations include diffusion computation costs and naive support-set generation, pointing to future work on scalable diffusion, smarter support sampling, and improved query-generation strategies.

Abstract

We propose Unsupervised Episode Generation method called Neighbors as Queries (NaQ) to solve the Few-Shot Node-Classification (FSNC) task by unsupervised Graph Meta-learning. Doing so enables full utilization of the information of all nodes in a graph, which is not possible in current supervised meta-learning methods for FSNC due to the label-scarcity problem. In addition, unlike unsupervised Graph Contrastive Learning (GCL) methods that overlook the downstream task to be solved at the training phase resulting in vulnerability to class imbalance of a graph, we adopt the episodic learning framework that allows the model to be aware of the downstream task format, i.e., FSNC. The proposed NaQ is a simple but effective unsupervised episode generation method that randomly samples nodes from a graph to make a support set, followed by similarity-based sampling of nodes to make the corresponding query set. Since NaQ is model-agnostic, any existing supervised graph meta-learning methods can be trained in an unsupervised manner, while not sacrificing much of their performance or sometimes even improving them. Extensive experimental results demonstrate the effectiveness of our proposed unsupervised episode generation method for graph meta-learning towards the FSNC task. Our code is available at: https://github.com/JhngJng/NaQ-PyTorch.

Figures (15)

• Figure 1: (a): Impact of the label-scarcity on supervised graph meta-learning methods ('Class%': a rate of available base classes during training, 'Label%': a rate of available labeled samples for each class). (b): Impact of the (randomly injected) label noise $p$ on supervised graph meta-learning methods. (c): Impact of the class imbalance ('Pareto' setting: we kept nodes for top-20% head classes, while keeping only 10 nodes for remaining classes; 'Extreme' setting: the only difference from the 'Pareto' setting is that we kept nodes only for top-5 head classes instead of top-20% classes). (5-way 1-shot)
• Figure 2: Overview of the NaQ-Feat.
• Figure 3: Result of applying NaQ-Feat and NaQ-Diff to existing graph meta-learning models (5-way 1-shot).
• Figure 4: Result of applying NaQ-Feat and NaQ-Diff to existing graph meta-learning models in higher way settings (Amazon-Clothing: 10-way 1-shot, Others: 20-way 1-shot).
• Figure 5: The t-SNE plot of tail-class embeddings (base-model: ProtoNet, NaQ: trained with 5-way 1-shot training episodes)

Theorems & Definitions (4)

• Definition 2.1: Supervised FSNC
• Definition 2.2: Unsupervised FSNC
• Definition 2.3: Episodic Learning
• Definition 3.1