Neural Temporal Point Processes for Forecasting Directional Relations in Evolving Hypergraphs

TL;DR

The paper addresses forecasting of directional, higher-order relationships in evolving hypergraphs by introducing DHyperNodeTPP, a neural temporal point process that operates in three stages: forecast node events, generate candidate hyperedges, and predict the true directed hyperedges. It leverages temporal node representations with a Memory Module, batch processing via a memory-based architecture, and a cross-attention-based Hyperedge Predictor to fuse dynamic and static information from both left and right hyperedges. The approach achieves substantial gains over state-of-the-art baselines in event-type and event-time prediction across multiple real-world datasets, demonstrating the value of modeling directionality and higher-order interactions in temporal networks. The work contributes a scalable framework, a novel directed hyperedge predictor, and five real-world datasets, with strong empirical evidence that directed hyperedge modeling yields improved predictive performance in complex temporal systems.

Abstract

Forecasting relations between entities is paramount in the current era of data and AI. However, it is often overlooked that real-world relationships are inherently directional, involve more than two entities, and can change with time. In this paper, we provide a comprehensive solution to the problem of forecasting directional relations in a general setting, where relations are higher-order, i.e., directed hyperedges in a hypergraph. This problem has not been previously explored in the existing literature. The primary challenge in solving this problem is that the number of possible hyperedges is exponential in the number of nodes at each event time. To overcome this, we propose a sequential generative approach that segments the forecasting process into multiple stages, each contingent upon the preceding stages, thereby reducing the search space involved in predictions of hyperedges. The first stage involves a temporal point process-based node event forecasting module that identifies the subset of nodes involved in an event. The second stage is a candidate generation module that predicts hyperedge sizes and adjacency vectors for nodes observing events. The final stage is a directed hyperedge predictor that identifies the truth by searching over the set of candidate hyperedges. To validate the effectiveness of our model, we compiled five datasets and conducted an extensive empirical study to assess each downstream task. Our proposed method achieves a performance gain of 32\% and 41\% compared to the state-of-the-art pairwise and hyperedge event forecasting models, respectively, for the event type prediction.

Figures (11)

• Figure 1: Bitcoin transactions (Txns) are modeled as a temporal directed hypergraph graph with eight nodes to represent addresses, and three hyperedges to represent transactions. Here, $t_i$ is the time with $t_3 > t_2 > t_1$, and a hyperedge is represented as a tuple $h_i = (h_i^r, h_i^\ell)$ with $h_i^r$ the right hyperedge are the senders' addresses and $h_i^\ell$ the left hyperedge are receiver addresses.
• Figure 2: Neural Architecture of DHyperNodeTPP: We calculate the temporal node representation $\mathbf{V}(t)$ by combining the entries from the Memory module with information from recent relations where the node is involved in the left and right hyperedges. These temporal node representations are given as input to forecast nodes where events occur, followed by candidate hyperedge generation. Then the hyperedge prediction decoder in Section \ref{['sec:architecture_of_link_predictor']} is used to find the observed hyperedges.
• Figure 3: Comparison of the performance of our directed and undirected model on different forecasting tasks. Figures \ref{['fig:enron_connectivity']}, \ref{['fig:enron_mrr']}, \ref{['fig:enron_mae']} are on Enron-Email dataset. Here, we can observe that representation from DHyperNodeTPP performs better than HyperNodeTPP for adjacency forecasting. Furthermore, DhyperNodeTPP performs better than HyperNodeTPP for hyperedge sizes greater than two for the event type prediction. For event time prediction, both models perform equally, as events are modeled on nodes, and for hyperedge size prediction, directed models perform better. Hence, we can learn better representation using direction information.
• Figure 4: Hyperedge predictor architecture
• Figure 5: Hyperedge size prediction