Deep Representation Learning for Forecasting Recursive and Multi-Relational Events in Temporal Networks

TL;DR

This work tackles forecasting of recursive, multi-relational, higher-order events in temporal networks using a Relational Recursive Hyperedge Temporal Point Process (RRHyperTPP). It combines a dynamic node representation learned via a two-stage update-and-drift mechanism with a hyperedge link-prediction decoder, and trains the model using noise-contrastive estimation to avoid intractable survival-function computations. Key contributions include (i) a deep architecture for dynamic representations on recursive hypergraphs, (ii) a hyperedge-based conditional intensity function parameterized by node and relation embeddings, (iii) a KDE-based noise distribution for efficient NCE training, and (iv) extensive experiments showing improvements over state-of-the-art baselines on depth-one and depth-two datasets. The approach advances temporal network forecasting by capturing complex hierarchies and relations, with practical impact for domains such as finance, communication networks, and event forecasting where higher-order interactions matter.

Abstract

Understanding relations arising out of interactions among entities can be very difficult, and predicting them is even more challenging. This problem has many applications in various fields, such as financial networks and e-commerce. These relations can involve much more complexities than just involving more than two entities. One such scenario is evolving recursive relations between multiple entities, and so far, this is still an open problem. This work addresses the problem of forecasting higher-order interaction events that can be multi-relational and recursive. We pose the problem in the framework of representation learning of temporal hypergraphs that can capture complex relationships involving multiple entities. The proposed model, \textit{Relational Recursive Hyperedge Temporal Point Process} (RRHyperTPP) uses an encoder that learns a dynamic node representation based on the historical interaction patterns and then a hyperedge link prediction-based decoder to model the occurrence of interaction events. These learned representations are then used for downstream tasks involving forecasting the type and time of interactions. The main challenge in learning from hyperedge events is that the number of possible hyperedges grows exponentially with the number of nodes in the network. This will make the computation of negative log-likelihood of the temporal point process expensive, as the calculation of survival function requires a summation over all possible hyperedges. In our work, we develop a noise contrastive estimation method to learn the parameters of our model, and we have experimentally shown that our models perform better than previous state-of-the-art methods for interaction forecasting.

Figures (6)

• Figure 1: Real world events represented as Relational Recursive Hyperedges. Here, there are source and target hyperedges with three relations inside them and relation types between them to indicate the nature of the interaction.
• Figure 2: $\mathrm{Loss}$ for RRHyperTPP and its variants on Enron dataset. We can observe that the proposed RRHyperTPP models perform better than their variants.
• Figure 3: The performance gain obtained due to the addition of classification-based noise contrastive loss mentioned Equation \ref{['eq:combined_nce_loss']}. Here, we can observe that models trained with supervised noise contrastive loss $\alpha>0$ have more AUC scores for interaction type prediction than models that do not ($\alpha=0$).
• Figure 4: Email exchange between a group of people is represented as a Relational Recursive Hyperedges depth. Here, the sender involves a single person, and multiple persons are in CCed and Receiver groups.
• Figure 5: Deep Neural Network Architecture of RRHyperTPP: We calculate the dynamic node representation $\mathbf{V} (t)$ by using Node Update (Section \ref{['sec:node_update']}) and Drift (Section \ref{['sec:drift']}) stages. The node Update stage is used to update the node embeddings after an interaction involving that node occurs, and the Drift stage is used to model the evolution of node embedding during the interevent period. The dynamic node representation is used by the hyperedge link prediction-based decoder to infer the conditional intensity function $\lambda_h (t)$.