Smoothed Graph Contrastive Learning via Seamless Proximity Integration

TL;DR

A Smoothed Graph Contrastive Learning model is presented, which leverages the geometric structure of augmented graphs to inject proximity information associated with positive/negative pairs in the contrastive loss, thus significantly regularizing the learning process.

Abstract

Graph contrastive learning (GCL) aligns node representations by classifying node pairs into positives and negatives using a selection process that typically relies on establishing correspondences within two augmented graphs. The conventional GCL approaches incorporate negative samples uniformly in the contrastive loss, resulting in the equal treatment of negative nodes, regardless of their proximity to the true positive. In this paper, we present a Smoothed Graph Contrastive Learning model (SGCL), which leverages the geometric structure of augmented graphs to inject proximity information associated with positive/negative pairs in the contrastive loss, thus significantly regularizing the learning process. The proposed SGCL adjusts the penalties associated with node pairs in contrastive loss by incorporating three distinct smoothing techniques that result in proximity-aware positives and negatives. To enhance scalability for large-scale graphs, the proposed framework incorporates a graph batch-generating strategy that partitions the given graphs into multiple subgraphs, facilitating efficient training in separate batches. Through extensive experimentation in the unsupervised setting on various benchmarks, particularly those of large scale, we demonstrate the superiority of our proposed framework against recent baselines.

Figures (4)

• Figure 1: An illustrative example of the efficacy of the smoothing approaches on a grid graph $\mathcal{G}$. We color the grid according to the node value. In the left grid, initial values of 1 are represented in yellow, whereas nodes with zero values are depicted in dark purple. Each smoothing approach modifies the values of the zero nodes according to neighboring information.
• Figure 2: In the general context of conventional contrastive learning approaches, for every anchor node $v_4^{(i)}$ in $\mathcal{G}^{(i)}$, a corresponding positive node $v_4^{(j)}$ exists in $\mathcal{G}^{(j)}$, with all other node pairs being negative (left image). Smoothing techniques, which leverages the geometry of graph $\mathcal{G}^{(j)}$, effectively extract neighboring node information of node $v_4^{(j)}$ and generate smoothed positive and negative pairs matrices $\tilde{\Pi}_{\text{pos}}^{(i,j)}$ and $\tilde{\Pi}_{\text{neg}}^{(i,j)}$ (right image).
• Figure 3: Comparison of mean disparity among graph encoders, illustrating that all SGCL variants consistently achieve lower values than the conventional GCL approach, which signifies a more effective self-supervised learning framework.
• Figure 4: Overview of the Proposed SGCL Model. The model first generates $|B|$ subgraphs and extracts two distinct views for each subgraph, denoted as $\hat{\mathcal{G}}_b^{(i)}$ and $\hat{\mathcal{G}}_b^{(j)}$. The GCN encoder is then employed to learn feature embeddings $\mathbf{H}_b^{(i)}$ and $\mathbf{H}_b^{(j)}$, respectively. Finally, the smoothed contrastive loss $\mathcal{L}_{\text{SGCL}}$ measures the agreement between these representations by utilizing $\tilde{\Pi}_{\text{pos}}^{(i,j)}$ and $\tilde{\Pi}_{\text{neg}}^{(i,j)}$.

Theorems & Definitions (5)

• Definition 2.1: Positive and negative set
• Definition 2.2: Smoothing process
• Definition 2.3: Dirichlet energy
• Lemma 2.1: Disparity
• Proposition 2.1