Deep-Graph-Sprints: Accelerated Representation Learning in Continuous-Time Dynamic Graphs

TL;DR

This paper introduces Deep-Graph-Sprints (DGS), a novel deep learning architecture designed for efficient representation learning on CTDGs with low-latency inference requirements that effectively bridges the gap between deep representation learning and low-latency application requirements for CTDGs.

Abstract

Continuous-time dynamic graphs (CTDGs) are essential for modeling interconnected, evolving systems. Traditional methods for extracting knowledge from these graphs often depend on feature engineering or deep learning. Feature engineering is limited by the manual and time-intensive nature of crafting features, while deep learning approaches suffer from high inference latency, making them impractical for real-time applications. This paper introduces Deep-Graph-Sprints (DGS), a novel deep learning architecture designed for efficient representation learning on CTDGs with low-latency inference requirements. We benchmark DGS against state-of-the-art (SOTA) feature engineering and graph neural network methods using five diverse datasets. The results indicate that DGS achieves competitive performance while inference speed improves between 4x and 12x compared to other deep learning approaches on our benchmark datasets. Our method effectively bridges the gap between deep representation learning and low-latency application requirements for CTDGs.

Figures (4)

• Figure 1: Schematic representation of the DGS architecture. The diagram illustrates the workflow from receiving new edge, through the generation of embeddings for nodes or edges, to the application of a neural network to generate a task specific score. Furthermore, the diagram elucidates the computation of gradients through the application of mixed-mode AD. Equation \ref{['eq:graph_sprints_formula_W']}, and Figure \ref{['fig:deep_graph_sprints_equation_example']} provide more details about the ER component.
• Figure 2: Schematic illustration of state calculation based on Equation \ref{['eq:graph_sprints_formula_W']}. This example demonstrates the computation of node state at time $t$ with a state size of $s=6$, three softmaxes ($m=3$), and thus two rows per softmax from the embedding matrix $W$ ($h=s/m=2$). The number of input features is $f=4$.
• Figure 3: Recurrent Training Process: This figure illustrates the steps involved over three successive timesteps, focusing on the derivative calculation for a single learnable parameter $\vec{\alpha}$ using a mixed-mode approach. It combines forward mode differentiation for the ER component with backpropagation for the neural network classifier. This methodology extends to update other parameters (i.e., $\vec{\beta}$, $W$). This hybrid approach enables an efficient solution that effectively captures long-term dependencies.
• Figure 4: Trade-off between AUC and runtime.