Rethinking Graph Contrastive Learning through Relative Similarity Preservation

TL;DR

The paper tackles the mismatch between graph data and absolute-similarity graph contrastive learning by uncovering a universal relative-similarity pattern: label consistency decays with structural distance as captured by multi-hop neighborhoods. It grounds this insight in random-walk theory, proving exponential convergence to a stationary distribution $\pi_j=\frac{\sum_{u:l(u)=j}\deg(u)}{\sum_v \deg(v)}$ and explaining how the second eigenvalue $\lambda_2$ drives either monotone or oscillatory decay in homophily and heterophily graphs. Building on this theory, the authors propose RelGCL, a principled framework preserving collective relative similarity via a log-sum loss $\mathcal{L}^{\text{in}}$ and two implementations, RelGCLPair and RelGCLList, each with an $\alpha$-threshold to reflect statistical rather than deterministic relations. Empirically, RelGCL achieves state-of-the-art results across 11 graphs (8 homophily, 3 heterophily), scales to large graphs like ogbn-arxiv, and demonstrates robustness across downstream tasks such as clustering and similarity search, validating the practical impact of the relative-similarity perspective for graph representation learning.

Abstract

Graph contrastive learning (GCL) has achieved remarkable success by following the computer vision paradigm of preserving absolute similarity between augmented views. However, this approach faces fundamental challenges in graphs due to their discrete, non-Euclidean nature -- view generation often breaks semantic validity and similarity verification becomes unreliable. Through analyzing 11 real-world graphs, we discover a universal pattern transcending the homophily-heterophily dichotomy: label consistency systematically diminishes as structural distance increases, manifesting as smooth decay in homophily graphs and oscillatory decay in heterophily graphs. We establish theoretical guarantees for this pattern through random walk theory, proving label distribution convergence and characterizing the mechanisms behind different decay behaviors. This discovery reveals that graphs naturally encode relative similarity patterns, where structurally closer nodes exhibit collectively stronger semantic relationships. Leveraging this insight, we propose RELGCL, a novel GCL framework with complementary pairwise and listwise implementations that preserve these inherent patterns through collective similarity objectives. Extensive experiments demonstrate that our method consistently outperforms 20 existing approaches across both homophily and heterophily graphs, validating the effectiveness of leveraging natural relative similarity over artificial absolute similarity.

Figures (7)

• Figure 1: Visual vs. Graph contrastive learning: (a) image views preserve semantics, (b) graph augmentation may alter properties, and (c) graph view similarity is hard to assess.
• Figure 2: Label consistency (the average proportion of neighbors sharing the same label as the anchor node) decay patterns: smooth monotonic decay in homophily graphs versus oscillatory decay in heterophily graphs, both exhibiting an overall diminishing trend.
• Figure 3: A philosophical comparison of absolute similarity and relative similarity in GCL. The right side presents the core idea of RelGCL.
• Figure 4: Results of RelGCL with neighborhood range changing as $k=1, 2, 3, 4$, respectively.
• Figure 5: Similarities between the anchor nodes' and associated n-th hop neighbors' representations.

Theorems & Definitions (9)

• Theorem 1: Label Distribution Convergence
• proof : Proof Sketch
• Corollary 1: Label Consistency Decay
• proof : Proof Sketch
• Proposition 1: Decay Pattern Characterization
• Lemma 1: Basic Properties
• proof
• proof : Proof of Theorem \ref{['thm:convergence']}
• proof : Proof of Corollary \ref{['cor:lc_decay']}