Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring Task Unification in Graph Representation Learning via Generative Approach

Yulan Hu, Sheng Ouyang, Zhirui Yang, Ge Chen, Junchen Wan, Xiao Wang, Yong Liu

TL;DR

GA^2E is proposed, a unified adversarially masked autoencoder capable of addressing the above challenges seamlessly and introduces an auxiliary discriminator to discern the authenticity between the reconstructed (generated) subgraph and the input subgraph, thus ensuring the robustness of the graph representation through adversarial training mechanisms.

Abstract

Graphs are ubiquitous in real-world scenarios and encompass a diverse range of tasks, from node-, edge-, and graph-level tasks to transfer learning. However, designing specific tasks for each type of graph data is often costly and lacks generalizability. Recent endeavors under the "Pre-training + Fine-tuning" or "Pre-training + Prompt" paradigms aim to design a unified framework capable of generalizing across multiple graph tasks. Among these, graph autoencoders (GAEs), generative self-supervised models, have demonstrated their potential in effectively addressing various graph tasks. Nevertheless, these methods typically employ multi-stage training and require adaptive designs, which on one hand make it difficult to be seamlessly applied to diverse graph tasks and on the other hand overlook the negative impact caused by discrepancies in task objectives between the different stages. To address these challenges, we propose GA^2E, a unified adversarially masked autoencoder capable of addressing the above challenges seamlessly. Specifically, GA^2E proposes to use the subgraph as the meta-structure, which remains consistent across all graph tasks (ranging from node-, edge-, and graph-level to transfer learning) and all stages (both during training and inference). Further, GA^2E operates in a \textbf{"Generate then Discriminate"} manner. It leverages the masked GAE to reconstruct the input subgraph whilst treating it as a generator to compel the reconstructed graphs resemble the input subgraph. Furthermore, GA^2E introduces an auxiliary discriminator to discern the authenticity between the reconstructed (generated) subgraph and the input subgraph, thus ensuring the robustness of the graph representation through adversarial training mechanisms. We validate GA^2E's capabilities through extensive experiments on 21 datasets across four types of graph tasks.

Exploring Task Unification in Graph Representation Learning via Generative Approach

TL;DR

GA^2E is proposed, a unified adversarially masked autoencoder capable of addressing the above challenges seamlessly and introduces an auxiliary discriminator to discern the authenticity between the reconstructed (generated) subgraph and the input subgraph, thus ensuring the robustness of the graph representation through adversarial training mechanisms.

Abstract

Graphs are ubiquitous in real-world scenarios and encompass a diverse range of tasks, from node-, edge-, and graph-level tasks to transfer learning. However, designing specific tasks for each type of graph data is often costly and lacks generalizability. Recent endeavors under the "Pre-training + Fine-tuning" or "Pre-training + Prompt" paradigms aim to design a unified framework capable of generalizing across multiple graph tasks. Among these, graph autoencoders (GAEs), generative self-supervised models, have demonstrated their potential in effectively addressing various graph tasks. Nevertheless, these methods typically employ multi-stage training and require adaptive designs, which on one hand make it difficult to be seamlessly applied to diverse graph tasks and on the other hand overlook the negative impact caused by discrepancies in task objectives between the different stages. To address these challenges, we propose GA^2E, a unified adversarially masked autoencoder capable of addressing the above challenges seamlessly. Specifically, GA^2E proposes to use the subgraph as the meta-structure, which remains consistent across all graph tasks (ranging from node-, edge-, and graph-level to transfer learning) and all stages (both during training and inference). Further, GA^2E operates in a \textbf{"Generate then Discriminate"} manner. It leverages the masked GAE to reconstruct the input subgraph whilst treating it as a generator to compel the reconstructed graphs resemble the input subgraph. Furthermore, GA^2E introduces an auxiliary discriminator to discern the authenticity between the reconstructed (generated) subgraph and the input subgraph, thus ensuring the robustness of the graph representation through adversarial training mechanisms. We validate GA^2E's capabilities through extensive experiments on 21 datasets across four types of graph tasks.
Paper Structure (24 sections, 10 equations, 6 figures, 8 tables, 2 algorithms)

This paper contains 24 sections, 10 equations, 6 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. RELATED WORKS
  3. PRELIMINARIES
  4. Necessary Background
  5. METHODOLOGY
  6. Meta-structure Reformulation
  7. The Design of $\text{GA}^{2}\text{E}$
  8. Why it Works ?
  9. EXPERIMENTS
  10. Main Results ($\mathcal{RQ}1$)
  11. Ablation Study ($\mathcal{RQ}2$)
  12. Parameter Analysis ($\mathcal{RQ}3$)
  13. CONCLUSION
  14. Motivation
  15. The "Pre-training + Fine-tuning" paradigm
  16. ...and 9 more sections

Figures (6)

  • Figure 1: The Overview of $\text{GA}^{2}\text{E}$. $\text{GA}^{2}\text{E}$ accepts arbitrary graphs as input and reformulates them into subgraphs. Training is conducted adversarially within $\text{GA}^{2}\text{E}$, where a masked GAE is adopted to reconstruct the input subgraph and generate the reconstructed graph. Additionally, a GNN Readout operates as a discriminator to determine the authenticity between the reformulated graph and the reconstructed graph.
  • Figure 2: The subgraph represents the meta-structure that contains node, edge and graph level knowledge.
  • Figure 3: Ablation studies of main components.
  • Figure 4: Impact of mask ratio.
  • Figure 5: Comparison of our approach with existing "pre-training + fine-tuning" paradigm
  • ...and 1 more figures

Theorems & Definitions (1)

  • Definition 1