Gaussian Embedding of Temporal Networks

TL;DR

This work addresses the challenge of representing continuous-time temporal networks with embeddings that quantify uncertainty in node trajectories. It introduces TGNE, a Bayesian approach that embeds nodes as piecewise Gaussian trajectories by combining a Gaussian Random Walk prior with variational inference within the Continuous-time Latent Position Model framework and edge likelihoods modeled by Poisson processes. The key contributions are (i) a scalable, uncertainty-aware temporal embedding method, (ii) a mean-field variational inference scheme for critical points, (iii) analysis of hyperparameter effects on uncertainty and reconstruction, and (iv) open-source implementation and comprehensive experiments demonstrating competitive reconstruction and meaningful uncertainty insights. The approach advances temporal network analysis by providing interpretable, uncertainty-aware embeddings suitable for tasks like reconstruction and anomaly detection in continuous time.

Abstract

Representing the nodes of continuous-time temporal graphs in a low-dimensional latent space has wide-ranging applications, from prediction to visualization. Yet, analyzing continuous-time relational data with timestamped interactions introduces unique challenges due to its sparsity. Merely embedding nodes as trajectories in the latent space overlooks this sparsity, emphasizing the need to quantify uncertainty around the latent positions. In this paper, we propose TGNE (\textbf{T}emporal \textbf{G}aussian \textbf{N}etwork \textbf{E}mbedding), an innovative method that bridges two distinct strands of literature: the statistical analysis of networks via Latent Space Models (LSM)\cite{Hoff2002} and temporal graph machine learning. TGNE embeds nodes as piece-wise linear trajectories of Gaussian distributions in the latent space, capturing both structural information and uncertainty around the trajectories. We evaluate TGNE's effectiveness in reconstructing the original graph and modelling uncertainty. The results demonstrate that TGNE generates competitive time-varying embedding locations compared to common baselines for reconstructing unobserved edge interactions based on observed edges. Furthermore, the uncertainty estimates align with the time-varying degree distribution in the network, providing valuable insights into the temporal dynamics of the graph. To facilitate reproducibility, we provide an open-source implementation of TGNE at \url{https://github.com/aida-ugent/tgne}.

Figures (6)

• Figure 1: Probabilistic Graphical Model summarizing the CLPM. ${\mathcal{T}}^{(k)} = \{(i_m, j_m, t_m)\in{\mathcal{T}}|t_m \in I_k\}$ is the history of interactions happening in the time interval $I_k=[\eta_{k-1}, \eta_k]$. $z^{(k)}$ are the snapshots of latent positions at time $\eta_k$. The chunks of history ${\mathcal{T}}^{(k)}$ are conditionally independent given the latent positions at the boundaries of the interval $I_k$, namely $z^{(k-1)}$ and $z^{(k)}$
• Figure 2: Resulting latent positions on the Stochastic Block Model. Uncertainty is represented by the size of the markers. In the first period, the nodes are divided into two communities (Circles and crosses). Then in the second one, node 0 becomes a triangle and forms its own community. During that transition, node 0's uncertainty increases, especially when using a less informative prior ($\tau=50.0$). Finally, the same node 0 becomes a cross.
• Figure 3: Log-log plot of the node-level uncertainty u(i,k) as a function of the average distance to the neighbors whithin the same interval, with ($\tau=50.0, K=15$).
• Figure 4: Edge Level Uncertainty
• Figure 5: Latent Positions obtained on the Toy Dataset, the Highschool Dataset and the MIT Reality Mining Dataset.

Theorems & Definitions (3)

• Remark 1
• Theorem 3.1
• Theorem 4.1