A Generative Graph Contrastive Learning Model with Global Signal

TL;DR

This work tackles noisy and uniformly weighted contrastive signals in graph contrastive learning by introducing CSG2L, a two-module framework. The SVD-directed augmentation (SVD-aug) extracts global structure by performing a low-rank reconstruction of the normalized adjacency $\tilde{A}$ as $\hat{A}=U_q S_q V_q^T$, using randomized ApproxSVD for efficiency, while the Local-Global Dependency Learning (LGDL) fuses local and global information and applies an adaptive reweighting strategy to emphasize hard sample pairs. An adaptive reweighting scheme guided by pseudo-labels and a reweighting function $R$ modifies the InfoNCE loss, and the overall objective combines cross-entropy with a contrastive term. Empirical results across six real-world datasets show consistent improvements over strong baselines and validate the framework's compatibility with various GNNs; ablations confirm the necessity of both SVD-aug and LGDL. Overall, CSG2L provides a robust pathway to leverage global graph signals in self-supervised learning for more discriminative node representations.

Abstract

Graph contrastive learning (GCL) has garnered significant attention recently since it learns complex structural information from graphs through self-supervised learning manner. However, prevalent GCL models may suffer from performance degradation due to inappropriate contrastive signals. Concretely, they commonly generate augmented views based on random perturbation, which leads to biased essential structures due to the introduction of noise. In addition, they assign equal weight to both hard and easy sample pairs, thereby ignoring the difference in importance of the sample pairs. To address these issues, this study proposes a novel Contrastive Signal Generative Framework for Accurate Graph Learning (CSG2L) with the following two-fold ideas: a) building a singular value decomposition (SVD)-directed augmented module (SVD-aug) to obtain the global interactions as well as avoiding the random noise perturbation; b) designing a local-global dependency learning module (LGDL) with an adaptive reweighting strategy which can differentiate the effects of hard and easy sample pairs. Extensive experiments on benchmark datasets demonstrate that the proposed CSG2L outperforms the state-of-art baselines. Moreover, CSG2L is compatible with a variety of GNNs.

Figures (3)

• Figure 1: a) The overall structure of CS$\mathrm{G}^2$L. It includes two components: b) SVD-aug module performs SVD-directed reconstruction of the original graph to obtain augmented graph. c) LGDL module inputs both graphs into a shared GNN, and further input the embedding of original graph into an MLP classifier to compute the classification loss. In addition, an adaptive reweighting strategy is introduced for contrastive learning. Finally, the overall objective function can be obtained.
• Figure 2: Hyper-parameter analysis results of CS$\mathrm{G}^2$Ls on three benchmark datasets.
• Figure 3: Ablation study results on three benchmark datasets.