Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LAMP: Learnable Meta-Path Guided Adversarial Contrastive Learning for Heterogeneous Graphs

Siqing Li, Jin-Duk Park, Wei Huang, Xin Cao, Won-Yong Shin, Zhiqiang Xu

TL;DR

The paper tackles the fragility of unsupervised heterogeneous graph contrastive learning caused by arbitrary meta-path selections. It introduces LAMP, a dual-view framework that combines a high-order meta-path view (processed via Learnable Meta-Path Guided Augmentation) with a local network-schema view, connected through a unified HGNN encoder and a max-min InfoNCE objective with a learnable meta-path weighting vector $\boldsymbol{\tau}$. Key contributions include (i) an integrated meta-path sub-graph approach with edge-pruning-based sparsification, (ii) a differentiable adversarial augmentation strategy that preserves meaningful structure, and (iii) extensive empirical validation showing superior robustness and accuracy across four HGB datasets, along with ablations and hyper-parameter analyses. The work advances robust, scalable HGCL for IR applications by reducing reliance on a single meta-path choice and enabling effective utilization of diverse meta-path information.

Abstract

Heterogeneous graph neural networks (HGNNs) have significantly propelled the information retrieval (IR) field. Still, the effectiveness of HGNNs heavily relies on high-quality labels, which are often expensive to acquire. This challenge has shifted attention towards Heterogeneous Graph Contrastive Learning (HGCL), which usually requires pre-defined meta-paths. However, our findings reveal that meta-path combinations significantly affect performance in unsupervised settings, an aspect often overlooked in current literature. Existing HGCL methods have considerable variability in outcomes across different meta-path combinations, thereby challenging the optimization process to achieve consistent and high performance. In response, we introduce \textsf{LAMP} (\underline{\textbf{L}}earn\underline{\textbf{A}}ble \underline{\textbf{M}}eta-\underline{\textbf{P}}ath), a novel adversarial contrastive learning approach that integrates various meta-path sub-graphs into a unified and stable structure, leveraging the overlap among these sub-graphs. To address the denseness of this integrated sub-graph, we propose an adversarial training strategy for edge pruning, maintaining sparsity to enhance model performance and robustness. \textsf{LAMP} aims to maximize the difference between meta-path and network schema views for guiding contrastive learning to capture the most meaningful information. Our extensive experimental study conducted on four diverse datasets from the Heterogeneous Graph Benchmark (HGB) demonstrates that \textsf{LAMP} significantly outperforms existing state-of-the-art unsupervised models in terms of accuracy and robustness.

LAMP: Learnable Meta-Path Guided Adversarial Contrastive Learning for Heterogeneous Graphs

TL;DR

The paper tackles the fragility of unsupervised heterogeneous graph contrastive learning caused by arbitrary meta-path selections. It introduces LAMP, a dual-view framework that combines a high-order meta-path view (processed via Learnable Meta-Path Guided Augmentation) with a local network-schema view, connected through a unified HGNN encoder and a max-min InfoNCE objective with a learnable meta-path weighting vector . Key contributions include (i) an integrated meta-path sub-graph approach with edge-pruning-based sparsification, (ii) a differentiable adversarial augmentation strategy that preserves meaningful structure, and (iii) extensive empirical validation showing superior robustness and accuracy across four HGB datasets, along with ablations and hyper-parameter analyses. The work advances robust, scalable HGCL for IR applications by reducing reliance on a single meta-path choice and enabling effective utilization of diverse meta-path information.

Abstract

Heterogeneous graph neural networks (HGNNs) have significantly propelled the information retrieval (IR) field. Still, the effectiveness of HGNNs heavily relies on high-quality labels, which are often expensive to acquire. This challenge has shifted attention towards Heterogeneous Graph Contrastive Learning (HGCL), which usually requires pre-defined meta-paths. However, our findings reveal that meta-path combinations significantly affect performance in unsupervised settings, an aspect often overlooked in current literature. Existing HGCL methods have considerable variability in outcomes across different meta-path combinations, thereby challenging the optimization process to achieve consistent and high performance. In response, we introduce \textsf{LAMP} (\underline{\textbf{L}}earn\underline{\textbf{A}}ble \underline{\textbf{M}}eta-\underline{\textbf{P}}ath), a novel adversarial contrastive learning approach that integrates various meta-path sub-graphs into a unified and stable structure, leveraging the overlap among these sub-graphs. To address the denseness of this integrated sub-graph, we propose an adversarial training strategy for edge pruning, maintaining sparsity to enhance model performance and robustness. \textsf{LAMP} aims to maximize the difference between meta-path and network schema views for guiding contrastive learning to capture the most meaningful information. Our extensive experimental study conducted on four diverse datasets from the Heterogeneous Graph Benchmark (HGB) demonstrates that \textsf{LAMP} significantly outperforms existing state-of-the-art unsupervised models in terms of accuracy and robustness.
Paper Structure (27 sections, 15 equations, 7 figures, 6 tables)

This paper contains 27 sections, 15 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Empirical Observations
  4. The Proposed Model: LAMP
  5. Problem Formulation
  6. Integrated Sub-graph based meta-path view
  7. Learnable Meta-Path Guided Augmentation
  8. Network Schema view
  9. Unified HGNN Encoder
  10. Node Residual:
  11. Edge Residual:
  12. Contrastive Optimization
  13. Experimental Evaluation
  14. Experimental Setup
  15. Datasets:
  16. ...and 12 more sections

Figures (7)

  • Figure 1: Comparing performance variability in node classification and illustrating standard deviation and min-max gaps across HGNN models (supervised HAN with 1/2 layers, unsupervised XGOAL, HeCo, DMGI, and our LAMP) using varied meta-path combinations.
  • Figure 2: A simplistic toy example derived from the ACM dataset: (a) Illustrates a Heterogeneous Graph. (b) Demonstrates three distinct meta-path sub-graphs associated with their respective meta-paths: PAP, PSP, and PAPAP. (c) Displays an integrated meta-path sub-graph that aggregates all the meta-path sub-graphs; its edge type embedding indicates which meta-paths are involved in each edge.
  • Figure 3: We generated a total of 26 distinct meta-path combinations using five predefined meta-paths: PAP, PSP, PTP, PPSP, and -PPSP. A flag "1" indicates the inclusion of a particular meta-path in the combination, whereas the absence of a meta-path is denoted by a flag "0". Each column on the right side of the table ranks the performance of these meta-path combinations for different models.
  • Figure 4: We calculated Jaccard Similarity and coverage ratio based on meta-path instances (edges) in meta-path sub-graphs.
  • Figure 5: Overall architecture of the proposed LAMP model. LAMP processes network schema view $G$ and meta-path graph $t(\hat{G})$, which supply local and high-order information, respectively. The adversarial training mechanism is aplied to enhance the robustness of the meta-path view, alongside the contrastive optimization strategy employed to minimize the discrepancy between the two views.
  • ...and 2 more figures