Simple Graph Contrastive Learning via Fractional-order Neural Diffusion Networks

TL;DR

This work tackles unsupervised node representation learning on graphs without data augmentations or negative samples by introducing FD GCL, a simple yet effective augmentation free contrastive framework based on fractional order neural diffusion. Two encoders governed by different order parameters generate distinct views that capture local and global graph information, and a regularized cosine mean loss ensures view diversity while mitigating collapse. The approach is underpinned by graph signal processing and fractional differential equation theory, with extensive experiments showing state of the art performance across both homophilic and heterophilic graphs and strong robustness to loss function choice. The results highlight the practical impact of memory aware diffusion dynamics for flexible and scalable graph representation learning.

Abstract

Graph Contrastive Learning (GCL) has recently made progress as an unsupervised graph representation learning paradigm. GCL approaches can be categorized into augmentation-based and augmentation-free methods. The former relies on complex data augmentations, while the latter depends on encoders that can generate distinct views of the same input. Both approaches may require negative samples for training. In this paper, we introduce a novel augmentation-free GCL framework based on graph neural diffusion models. Specifically, we utilize learnable encoders governed by Fractional Differential Equations (FDE). Each FDE is characterized by an order parameter of the differential operator. We demonstrate that varying these parameters allows us to produce learnable encoders that generate diverse views, capturing either local or global information, for contrastive learning. Our model does not require negative samples for training and is applicable to both homophilic and heterophilic datasets. We demonstrate its effectiveness across various datasets, achieving state-of-the-art performance.

Figures (11)

• Figure 1: The t-SNE visualizations of node features from a single class, generated by encoders with different FDE order parameters. For comparison, features are linearly translated to align class averages. The datasets used are Cora (homophilic) and Wisconsin (heterophilic). The visualizations demonstrate that the two encoders produce embeddings with distinct characteristics. A smaller $\alpha$ produces features with a concentrated core, while features generated by a larger $\alpha$ are more evenly spaced. Additional results for other label classes are provided in \ref{['sec:mnr']}.
• Figure 2: The PCA components of features for different datasets and choices of FDE order parameters. We see that for the small order ($\alpha_1$), the bar chart is comparatively more spread out, which prevents dimension collapse.
• Figure 3: The variation in the ratio $r_c$ during training. Each curve represents a label class. The ratio $r_c$ for the input features is shown at epoch $0$. We see that the ratio generally increases at the beginning of the training and stabilizes. This suggests that the encoders indeed have good clustering capabilities (as compared with the input).
• Figure 4: The overall proposed contrastive learning framework of FD-GCL. We choose $\alpha_1<\alpha_2$ for the encoders.
• Figure 5: Accuracy vs. training epochs for various loss functions on the Cora and Wisconsin datasets.

Theorems & Definitions (1)

• proof