CLDG: Contrastive Learning on Dynamic Graphs

TL;DR

CLDG tackles unsupervised representation learning on dynamic graphs by introducing temporal translation invariance and a timespan view sampling layer to extract temporally persistent signals. It processes multiple time-spanned views with a shared-weight encoder, a simple readout, and a two-layer projection head, optimizing local and global contrastive losses via an InfoNCE objective with temperature $\tau$. The approach achieves state-of-the-art performance among unsupervised methods across seven real-world datasets and surpasses several supervised baselines in some cases, while delivering major reductions in parameter count and training time. This framework is highly scalable and flexible with respect to encoder choice and time modeling, providing a practical tool for large-scale dynamic-graph representation learning.

Abstract

The graph with complex annotations is the most potent data type, whose constantly evolving motivates further exploration of the unsupervised dynamic graph representation. One of the representative paradigms is graph contrastive learning. It constructs self-supervised signals by maximizing the mutual information between the statistic graph's augmentation views. However, the semantics and labels may change within the augmentation process, causing a significant performance drop in downstream tasks. This drawback becomes greatly magnified on dynamic graphs. To address this problem, we designed a simple yet effective framework named CLDG. Firstly, we elaborate that dynamic graphs have temporal translation invariance at different levels. Then, we proposed a sampling layer to extract the temporally-persistent signals. It will encourage the node to maintain consistent local and global representations, i.e., temporal translation invariance under the timespan views. The extensive experiments demonstrate the effectiveness and efficiency of the method on seven datasets by outperforming eight unsupervised state-of-the-art baselines and showing competitiveness against four semi-supervised methods. Compared with the existing dynamic graph method, the number of model parameters and training time is reduced by an average of 2,001.86 times and 130.31 times on seven datasets, respectively.

Figures (5)

• Figure 1: Illustration of our basic idea. In the dataset used in this paper (details of the datasets are demonstrated in Section \ref{['dataset']}), an interesting observation is that the semantics and labels learned by the same node tend to be similar no matter what encoder is used at different timespans of the dynamic graph. We refer to this observation as temporal translation invariance. Based on temporal translation invariance, our basic idea is that the features of the node A, on different timespan should be similar and pull apart the features of other nodes, such as B.
• Figure 2: The architecture of the CLDG. The core of CLDG is implemented by maintaining local or global temporal translation invariance. Given an input graph, we first sample multiple views through a timespan view sampling layer. Subsequently, the multi-views are fed into a shared weight encoder to generate node embeddings, and the node neighborhood embedding is generated through the readout function. Then, node and neighborhood embeddings are fed into a projection head that maps the embeddings into the space of the contrastive loss. Finally, the contrastive approach is used to maintain the temporal translation invariance of the local and the global with a batch size of $N$.
• Figure 3: Four candidate timespan view sampling strategies of CLDG.
• Figure 4: Projection head of CLDG.
• Figure 5: Parameter sensitivity of CLDG. Effect of epoch, batch size, output dimension and layers on the node classification.