Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spectral-Aware Augmentation for Enhanced Graph Representation Learning

Kaiqi Yang, Haoyu Han, Wei Jin, Hui Liu

TL;DR

This work identifies a mismatch between common graph contrastive-learning augmentations and the spectral distribution of task-relevant information. It proposes GASSER, a spectral-aware augmentation framework that selectively perturbs a chosen set of eigenpairs, reconstructs an augmented adjacency, and applies selective edge flipping to generate more informative views. The authors provide theoretical results and extensive experiments showing that GASSER improves performance across both homophilic and heterophilic graphs and can integrate with multiple GCL frameworks while offering robustness to structure attacks. The approach advances practical graph representation learning by aligning augmentation with spectral properties and graph properties such as homophily, enabling more effective downstream tasks.

Abstract

Graph Contrastive Learning (GCL) has demonstrated remarkable effectiveness in learning representations on graphs in recent years. To generate ideal augmentation views, the augmentation generation methods should preserve essential information while discarding less relevant details for downstream tasks. However, current augmentation methods usually involve random topology corruption in the spatial domain, which fails to adequately address information spread across different frequencies in the spectral domain. Our preliminary study highlights this issue, demonstrating that spatial random perturbations impact all frequency bands almost uniformly. Given that task-relevant information typically resides in specific spectral regions that vary across graphs, this one-size-fits-all approach can pose challenges. We argue that indiscriminate spatial random perturbation might unintentionally weaken task-relevant information, reducing its effectiveness. To tackle this challenge, we propose applying perturbations selectively, focusing on information specific to different frequencies across diverse graphs. In this paper, we present GASSER, a model that applies tailored perturbations to specific frequencies of graph structures in the spectral domain, guided by spectral hints. Through extensive experimentation and theoretical analysis, we demonstrate that the augmentation views generated by GASSER are adaptive, controllable, and intuitively aligned with the homophily ratios and spectrum of graph structures.

Spectral-Aware Augmentation for Enhanced Graph Representation Learning

TL;DR

This work identifies a mismatch between common graph contrastive-learning augmentations and the spectral distribution of task-relevant information. It proposes GASSER, a spectral-aware augmentation framework that selectively perturbs a chosen set of eigenpairs, reconstructs an augmented adjacency, and applies selective edge flipping to generate more informative views. The authors provide theoretical results and extensive experiments showing that GASSER improves performance across both homophilic and heterophilic graphs and can integrate with multiple GCL frameworks while offering robustness to structure attacks. The approach advances practical graph representation learning by aligning augmentation with spectral properties and graph properties such as homophily, enabling more effective downstream tasks.

Abstract

Graph Contrastive Learning (GCL) has demonstrated remarkable effectiveness in learning representations on graphs in recent years. To generate ideal augmentation views, the augmentation generation methods should preserve essential information while discarding less relevant details for downstream tasks. However, current augmentation methods usually involve random topology corruption in the spatial domain, which fails to adequately address information spread across different frequencies in the spectral domain. Our preliminary study highlights this issue, demonstrating that spatial random perturbations impact all frequency bands almost uniformly. Given that task-relevant information typically resides in specific spectral regions that vary across graphs, this one-size-fits-all approach can pose challenges. We argue that indiscriminate spatial random perturbation might unintentionally weaken task-relevant information, reducing its effectiveness. To tackle this challenge, we propose applying perturbations selectively, focusing on information specific to different frequencies across diverse graphs. In this paper, we present GASSER, a model that applies tailored perturbations to specific frequencies of graph structures in the spectral domain, guided by spectral hints. Through extensive experimentation and theoretical analysis, we demonstrate that the augmentation views generated by GASSER are adaptive, controllable, and intuitively aligned with the homophily ratios and spectrum of graph structures.
Paper Structure (25 sections, 3 theorems, 9 equations, 4 figures, 3 tables)

This paper contains 25 sections, 3 theorems, 9 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary Study
  4. Task-Relevant Information from the Spectral Domain
  5. Impact of Random Spatial Augmentation on Graph Spectrum
  6. The Proposed Framework
  7. Selective Perturbation on Eigenpairs
  8. Adjacency Matrix Reconstruction
  9. Selective Edge Flipping
  10. Efficiency Improvement
  11. Model Training
  12. Theorem Proofs
  13. Proof of Theorem \ref{['thm:1']}
  14. Proof of Theorem \ref{['thm:2']}
  15. Proof of Theorem \ref{['thm:3']}
  16. ...and 10 more sections

Key Result

Theorem 1

Given two label vectors ${\bf y}\in\{0,1\}^{ n}$ and ${\bf \hat{y}}\in\{0,1\}^{ n}$ which are defined on $\mathcal{G}$, we can decompose them into ${\bf y} = \sum_{i=0}^{ n-1} c_i{\bf u}_i$ and ${\bf \hat{y}} = \sum_{i=0}^{ n-1} \hat{c}_i{\bf u}_i$. We denote their homophily as $h_1$ and $h_2$, resp

Figures (4)

  • Figure 1: Normalized distance between the grouped decomposed components of Laplacian matrixes of the original and augmented view with random 20% edge insertion.
  • Figure 2: An Overview of GASSER.
  • Figure 3: Node classification performance in attack setting on four datasets. RAND stands for random attack strategies and DICE stands for DICE dice; the following numbers indicate attack budgets.
  • Figure 4: Node classification performance of ablation study on four datasets. For each dataset, the results are of GASSER-Full, GASSER-W, and GASSER-B respectively.

Theorems & Definitions (3)

  • Theorem 1
  • Theorem 2
  • Theorem 3