Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Revisiting and Benchmarking Graph Autoencoders: A Contrastive Learning Perspective

Jintang Li, Ruofan Wu, Yuchang Zhu, Huizhe Zhang, Xinzhou Jin, Guibin Zhang, Zulun Zhu, Zibin Zheng, Liang Chen

TL;DR

The paper argues that GAEs can be understood as contrastive learners over paired subgraph views and introduces lrGAE, a five-component framework (augmentations, contrastive views, encoder/decoder networks, contrastive loss, and negative samples) that unifies GAEs with graph contrastive learning. It formalizes the connection via alignment $\ extcal{L}_{alignment}$ and uniformity $\textcal{L}_{uniformity}$ losses and presents a comprehensive GAE benchmark implemented within the lrGAE framework. The authors provide extensive experiments across node/edge/graph tasks and heterogeneous graphs, showing that lrGAE variants and masked GAEs achieve competitive or superior performance to state-of-the-art baselines, with insights from ablations on augmentations, encoders, losses, and negatives. Public code and tutorials enable reproducibility and fair comparisons, positioning lrGAE as a practical platform for advancing graph self-supervised learning.

Abstract

Graph autoencoders (GAEs) are self-supervised learning models that can learn meaningful representations of graph-structured data by reconstructing the input graph from a low-dimensional latent space. Over the past few years, GAEs have gained significant attention in academia and industry. In particular, the recent advent of GAEs with masked autoencoding schemes marks a significant advancement in graph self-supervised learning research. While numerous GAEs have been proposed, the underlying mechanisms of GAEs are not well understood, and a comprehensive benchmark for GAEs is still lacking. In this work, we bridge the gap between GAEs and contrastive learning by establishing conceptual and methodological connections. We revisit the GAEs studied in previous works and demonstrate how contrastive learning principles can be applied to GAEs. Motivated by these insights, we introduce lrGAE (left-right GAE), a general and powerful GAE framework that leverages contrastive learning principles to learn meaningful representations. Our proposed lrGAE not only facilitates a deeper understanding of GAEs but also sets a new benchmark for GAEs across diverse graph-based learning tasks. The source code for lrGAE, including the baselines and all the code for reproducing the results, is publicly available at https://github.com/EdisonLeeeee/lrGAE.

Revisiting and Benchmarking Graph Autoencoders: A Contrastive Learning Perspective

TL;DR

The paper argues that GAEs can be understood as contrastive learners over paired subgraph views and introduces lrGAE, a five-component framework (augmentations, contrastive views, encoder/decoder networks, contrastive loss, and negative samples) that unifies GAEs with graph contrastive learning. It formalizes the connection via alignment and uniformity losses and presents a comprehensive GAE benchmark implemented within the lrGAE framework. The authors provide extensive experiments across node/edge/graph tasks and heterogeneous graphs, showing that lrGAE variants and masked GAEs achieve competitive or superior performance to state-of-the-art baselines, with insights from ablations on augmentations, encoders, losses, and negatives. Public code and tutorials enable reproducibility and fair comparisons, positioning lrGAE as a practical platform for advancing graph self-supervised learning.

Abstract

Graph autoencoders (GAEs) are self-supervised learning models that can learn meaningful representations of graph-structured data by reconstructing the input graph from a low-dimensional latent space. Over the past few years, GAEs have gained significant attention in academia and industry. In particular, the recent advent of GAEs with masked autoencoding schemes marks a significant advancement in graph self-supervised learning research. While numerous GAEs have been proposed, the underlying mechanisms of GAEs are not well understood, and a comprehensive benchmark for GAEs is still lacking. In this work, we bridge the gap between GAEs and contrastive learning by establishing conceptual and methodological connections. We revisit the GAEs studied in previous works and demonstrate how contrastive learning principles can be applied to GAEs. Motivated by these insights, we introduce lrGAE (left-right GAE), a general and powerful GAE framework that leverages contrastive learning principles to learn meaningful representations. Our proposed lrGAE not only facilitates a deeper understanding of GAEs but also sets a new benchmark for GAEs across diverse graph-based learning tasks. The source code for lrGAE, including the baselines and all the code for reproducing the results, is publicly available at https://github.com/EdisonLeeeee/lrGAE.

Paper Structure

This paper contains 13 sections, 1 theorem, 9 equations, 4 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Graph autoencoders: generative yet contrastive
  4. lrGAE: design space for GAEs from contrastive learning perspective
  5. Experiments
  6. Main results
  7. Conclusion
  8. lrGAE framework and implementations
  9. Discussions
  10. Reproducibility
  11. Experimental results
  12. Ablation studies
  13. Remarks.

Key Result

Lemma 3.1

Under mild conditions, the GAE loss (Eq. eq:gae2) is lower bounded by an alignment loss which is induced by the inner product:

Figures (4)

  • Figure 1: Technical comparison between GAE gae, MaskGAE maskgae, and GraphMAE graphmae from a contrastive learning perspective.
  • Figure 2: Illustration of seven possible cases of lrGAE. We vary the graph views ($A$ and $B$), receptive fields ($l$ and $r$), as well as node pairs ($v$ and $u$) to implement different variants of lrGAE with different contrastive views.
  • Figure 3: Ablation on three lrGAE variants with different augmentation (masking) strategies, encoder networks, contrastive loss, and negative sampling strategies.
  • Figure : PyTorch pytorch style pseudocode for lrGAE.

Theorems & Definitions (1)

  • Lemma 3.1