Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Unsupervised Graph Few-shot Learning via Set Functions and Optimal Transport

Yonghao Liu, Fausto Giunchiglia, Ximing Li, Lan Huang, Xiaoyue Feng, Renchu Guan

TL;DR

STAR tackles unsupervised graph few-shot learning by combining permutation-invariant set representations with optimal-transport-based distribution calibration, built on graph contrastive learning for both instance- and set-level features. By jointly optimizing instance- and set-level objectives and then transporting the support distribution to align with the query distribution at test time, STAR mitigates distribution shift and enhances task-relevant information capture. Theoretical results show increased information content and tighter generalization bounds for the joint representation, while extensive experiments across diverse graphs demonstrate state-of-the-art performance and robustness. This approach reduces dependence on labeled base classes and improves generalization for real-world, unlabeled pretraining scenarios.

Abstract

Graph few-shot learning has garnered significant attention for its ability to rapidly adapt to downstream tasks with limited labeled data, sparking considerable interest among researchers. Recent advancements in graph few-shot learning models have exhibited superior performance across diverse applications. Despite their successes, several limitations still exist. First, existing models in the meta-training phase predominantly focus on instance-level features within tasks, neglecting crucial set-level features essential for distinguishing between different categories. Second, these models often utilize query sets directly on classifiers trained with support sets containing only a few labeled examples, overlooking potential distribution shifts between these sets and leading to suboptimal performance. Finally, previous models typically require necessitate abundant labeled data from base classes to extract transferable knowledge, which is typically infeasible in real-world scenarios. To address these issues, we propose a novel model named STAR, which leverages Set funcTions and optimAl tRansport for enhancing unsupervised graph few-shot learning. Specifically, STAR utilizes expressive set functions to obtain set-level features in an unsupervised manner and employs optimal transport principles to align the distributions of support and query sets, thereby mitigating distribution shift effects. Theoretical analysis demonstrates that STAR can capture more task-relevant information and enhance generalization capabilities. Empirically, extensive experiments across multiple datasets validate the effectiveness of STAR. Our code can be found here.

Enhancing Unsupervised Graph Few-shot Learning via Set Functions and Optimal Transport

TL;DR

STAR tackles unsupervised graph few-shot learning by combining permutation-invariant set representations with optimal-transport-based distribution calibration, built on graph contrastive learning for both instance- and set-level features. By jointly optimizing instance- and set-level objectives and then transporting the support distribution to align with the query distribution at test time, STAR mitigates distribution shift and enhances task-relevant information capture. Theoretical results show increased information content and tighter generalization bounds for the joint representation, while extensive experiments across diverse graphs demonstrate state-of-the-art performance and robustness. This approach reduces dependence on labeled base classes and improves generalization for real-world, unlabeled pretraining scenarios.

Abstract

Graph few-shot learning has garnered significant attention for its ability to rapidly adapt to downstream tasks with limited labeled data, sparking considerable interest among researchers. Recent advancements in graph few-shot learning models have exhibited superior performance across diverse applications. Despite their successes, several limitations still exist. First, existing models in the meta-training phase predominantly focus on instance-level features within tasks, neglecting crucial set-level features essential for distinguishing between different categories. Second, these models often utilize query sets directly on classifiers trained with support sets containing only a few labeled examples, overlooking potential distribution shifts between these sets and leading to suboptimal performance. Finally, previous models typically require necessitate abundant labeled data from base classes to extract transferable knowledge, which is typically infeasible in real-world scenarios. To address these issues, we propose a novel model named STAR, which leverages Set funcTions and optimAl tRansport for enhancing unsupervised graph few-shot learning. Specifically, STAR utilizes expressive set functions to obtain set-level features in an unsupervised manner and employs optimal transport principles to align the distributions of support and query sets, thereby mitigating distribution shift effects. Theoretical analysis demonstrates that STAR can capture more task-relevant information and enhance generalization capabilities. Empirically, extensive experiments across multiple datasets validate the effectiveness of STAR. Our code can be found here.
Paper Structure (28 sections, 3 theorems, 11 equations, 4 figures, 6 tables, 1 algorithm)

This paper contains 28 sections, 3 theorems, 11 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary
  4. Method
  5. Graph Contrastive Learning for Instance-level Features
  6. Graph Contrastive Learning for Set-level Features
  7. Optimal Transport for Distribution Calibration
  8. Theoretical Analysis
  9. Experiment
  10. Dataset
  11. Baseline
  12. Implementation Details
  13. Results
  14. Model Performance
  15. Ablation Study
  16. ...and 13 more sections

Key Result

Theorem 5.1

For the classification task $\mathrm{T}$, let $\mathrm{Z}$ denotes the node representations obtained under our training objective, and $\tilde{\mathrm{H}}$ and $\tilde{\mathrm{S}}$ denote the node representations obtained individually under the instance-level and set-level loss, respectively. Then w where $\mathrm{I}(\cdot;\cdot)$ is the mutual information.

Figures (4)

  • Figure 1: Data Distributions of support and query sets for two datasets.
  • Figure 2: The overall framework of our model.
  • Figure 3: (a): Distribution of support and query set after performing optimal transport. (b): Model performance varies with the value of $k$ in top-$k$ across all datasets.
  • Figure 4: Model performance varies with epochs across two datasets.

Theorems & Definitions (3)

  • Theorem 5.1
  • Corollary 5.2
  • Theorem 5.3