Contrastive Representation Learning for Dynamic Link Prediction in Temporal Networks

TL;DR

Temporal networks require representations that capture both structure and evolution. We introduce teneNCE, a self-supervised framework that learns dynamic representations from discrete-time snapshot graphs using a five-component model (Encoder, Update via GGRU, Decoder, LinkPredictor, PredictiveEncoder) and a multi-objective loss L = L_pred + $\alpha$L_recon + $\beta$L_cpc, where CPC is applied at both node and graph levels to forecast future topology. Empirical results on Enron, COLAB, and Facebook show teneNCE achieving superior dynamic link prediction performance compared to baselines, with ablations highlighting the benefit of incorporating reconstruction and both local/global CPC terms. The work demonstrates that combining predictive, reconstructive, and self-supervised contrastive signals yields richer temporal representations and improved forecasting, with practical implications for downstream analytics and decision-making on evolving networks.

Abstract

Evolving networks are complex data structures that emerge in a wide range of systems in science and engineering. Learning expressive representations for such networks that encode their structural connectivity and temporal evolution is essential for downstream data analytics and machine learning applications. In this study, we introduce a self-supervised method for learning representations of temporal networks and employ these representations in the dynamic link prediction task. While temporal networks are typically characterized as a sequence of interactions over the continuous time domain, our study focuses on their discrete-time versions. This enables us to balance the trade-off between computational complexity and precise modeling of the interactions. We propose a recurrent message-passing neural network architecture for modeling the information flow over time-respecting paths of temporal networks. The key feature of our method is the contrastive training objective of the model, which is a combination of three loss functions: link prediction, graph reconstruction, and contrastive predictive coding losses. The contrastive predictive coding objective is implemented using infoNCE losses at both local and global scales of the input graphs. We empirically show that the additional self-supervised losses enhance the training and improve the model's performance in the dynamic link prediction task. The proposed method is tested on Enron, COLAB, and Facebook datasets and exhibits superior results compared to existing models.

Figures (6)

• Figure 1: (a) Illustration of a temporal network as a sequence of pairwise interactions between system entities over a continuous time interval. (b) The discrete-time snapshot sequence represents the temporal network. The discretization operation divides the temporal network’s time domain into equal-length intervals $\Delta t$ and projects the interactions within each time interval to a static graph.
• Figure 2: (a) Illustration of the teneNCE model architecture. The model processes a sequence of snapshot graphs. The main components of the teneNCE model include an Encoder for embedding each static graph in the sequence and an Update component that recursively updates the state representations of each node across time steps. During the forward pass of the model in the training process, at time step $k$, the updated node states $\mathbf{S}_k$ are used to compute the (1) reconstruction loss, (2) prediction loss, and, (3) contrastive predictive coding loss. (b) The prediction modules include a Decoder for reconstructing the static graph at each time step and a LinkPredictor for predicting the graph's structure at the next time step. (c) An overview of the CPC loss module, which consists of a LocalPredictiveEncoder and a GlobalPredictiveEncoder for predicting the future structural embeddings of the graph based on the node states, for time steps $k+1, \dots, N$. Additionally, the $\textrm{ReadOut}(.)$ function aggregates the node-level embeddings into the graph-level representation.
• Figure 3: Illustration of positive and negative sample pairs for local and global infoNCE losses. This example depicts positive and negative pairs for node the $v_2$ and graph $G_k$, corresponding to local and global losses. For localNCE, different negative samples defined in Eq.\ref{['eq:negative_samples']} are colored orange, pink, and blue; for globalNCE, negative samples are colored pink.
• Figure 4: Visualization of the comparison between the temporal correlation coefficient values for the datasets in \ref{['subsec:datasets']} and their corresponding randomized versions. We have used two randomization models, namely Randomly Permuted Times (RP) and Randomized Edges (RE). For each null model, we have sampled 100 randomized versions. The box plots summarize the distribution of temporal correlation coefficients of the randomized samples. The figure demonstrates that the null models have a lower temporal correlation than the original data. This suggests that capturing temporal information of dynamic graphs is essential for effectively modeling such data.
• Figure 5: The density values of snapshot sequences over time. The figure shows the variation of each graph's structure over time.

Theorems & Definitions (2)

• Definition 3.1: Temporal network
• Definition 3.2: Snapshot sequence