Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explanation-Preserving Augmentation for Semi-Supervised Graph Representation Learning

Zhuomin Chen, Jingchao Ni, Hojat Allah Salehi, Xu Zheng, Esteban Schafir, Farhad Shirani, Dongsheng Luo

TL;DR

It is demonstrated theoretically, using an analytical example, and through extensive experiments on a variety of benchmark datasets, that EPA-GRL outperforms the state-of-the-art (SOTA) GRL methods that use semantics-agnostic augmentations.

Abstract

Self-supervised graph representation learning (GRL) typically generates paired graph augmentations from each graph to infer similar representations for augmentations of the same graph, but distinguishable representations for different graphs. While effective augmentation requires both semantics-preservation and data-perturbation, most existing GRL methods focus solely on data-perturbation, leading to suboptimal solutions. To fill the gap, in this paper, we propose a novel method, Explanation-Preserving Augmentation (EPA), which leverages graph explanation for semantics-preservation. EPA first uses a small number of labels to train a graph explainer, which infers the subgraphs that explain the graph's label. Then these explanations are used for generating semantics-preserving augmentations for boosting self-supervised GRL. Thus, the entire process, namely EPA-GRL, is semi-supervised. We demonstrate theoretically, using an analytical example, and through extensive experiments on a variety of benchmark datasets, that EPA-GRL outperforms the state-of-the-art (SOTA) GRL methods that use semantics-agnostic augmentations. The code is available at https://github.com/realMoana/EPA-GRL.

Explanation-Preserving Augmentation for Semi-Supervised Graph Representation Learning

TL;DR

It is demonstrated theoretically, using an analytical example, and through extensive experiments on a variety of benchmark datasets, that EPA-GRL outperforms the state-of-the-art (SOTA) GRL methods that use semantics-agnostic augmentations.

Abstract

Self-supervised graph representation learning (GRL) typically generates paired graph augmentations from each graph to infer similar representations for augmentations of the same graph, but distinguishable representations for different graphs. While effective augmentation requires both semantics-preservation and data-perturbation, most existing GRL methods focus solely on data-perturbation, leading to suboptimal solutions. To fill the gap, in this paper, we propose a novel method, Explanation-Preserving Augmentation (EPA), which leverages graph explanation for semantics-preservation. EPA first uses a small number of labels to train a graph explainer, which infers the subgraphs that explain the graph's label. Then these explanations are used for generating semantics-preserving augmentations for boosting self-supervised GRL. Thus, the entire process, namely EPA-GRL, is semi-supervised. We demonstrate theoretically, using an analytical example, and through extensive experiments on a variety of benchmark datasets, that EPA-GRL outperforms the state-of-the-art (SOTA) GRL methods that use semantics-agnostic augmentations. The code is available at https://github.com/realMoana/EPA-GRL.

Paper Structure

This paper contains 39 sections, 2 theorems, 15 equations, 11 figures, 10 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Notations and Problem Formulation
  4. The Proposed Method
  5. Pre-Training GNN Explainer
  6. Explanation-Preserving Augmentation Enhanced Graph Representation Learning
  7. Theoretical Analysis
  8. Experiments
  9. Experimental Setup
  10. Performance Analysis of Augmentation Methods
  11. Experiments on Synthetic and Large Datasets
  12. Ablation Study
  13. Impact of Explanation Quality on Model Performance
  14. Additional Analysis
  15. Conclusion
  16. ...and 24 more sections

Key Result

Theorem 1

In the modified BA-2Motifs classification task described above, consider the ECLs $f_\text{enc}^{sa}$ and $f_\text{enc}^{sp}$, which correspond to the semantic-agnostic augmentation $P^{sa}_{G'|G}$ and the semantic-preserving augmentation $P^{sp}_{G'|G}$, respectively, with edge-drop probability $p

Figures (11)

  • Figure 1: Semantics-preserving ability of different augmentations.
  • Figure 2: The Architecture of the proposed EPA-GRL method. We first pretrain a GNN model $f_{\text{pt}}(\cdot)$ and its explainer $\Psi(\cdot)$ with a small number of labeled training samples. Then in the GRL step, we use the frozen explainer $\Psi(\cdot)$ to produce augmented graphs to train a GNN encoder $f_\text{enc}(\cdot)$ and a projection head $f_\text{pro}(\cdot)$ with a contrastive loss. The output of the GNN encoder $f_\text{enc}(\cdot)$ will be used as graph representations.
  • Figure 3: Exemplar graphs in modified BA-2motifs.
  • Figure 4: Accuracy with different numbers of (a) # pre-training samples and (b) # downstream training samples.
  • Figure 5: Correlation between $Fid_{\alpha_1,{+}}$, $Fid_{\alpha_2,{-}}$, and Accuracy on MUTAG dataset. The value $r$ represents the Pearson correlation coefficient. Statistical significance is denoted by $\ast$, with $\ast\ast\ast$ indicating a p-value of $p \leq 0.01$ for testing non-correlation.
  • ...and 6 more figures

Theorems & Definitions (3)

  • Theorem 1
  • Definition 1: Empirical Contrastive Learner
  • Theorem 2