Higher-order Structure Boosts Link Prediction on Temporal Graphs

TL;DR

This paper tackles the limitation of traditional temporal graph neural networks that focus on pairwise interactions by introducing HTGN, a model that integrates hypergraph representations to capture higher-order group interactions. HTGN comprises a higher-order structure memory module and a hypergraph embedding module, enabling dynamic memory updates for hyperedges and hyperedge-based node embeddings for more expressive temporal learning. The authors prove HTGN is more expressive than conventional pairwise TGNNs and demonstrate superior dynamic link prediction performance across ten real-world datasets, while achieving substantial memory efficiency (up to 50% reduction) compared to baselines. The work suggests that leveraging higher-order structures can significantly boost both the predictive power and scalability of temporal graph models, pointing toward broader temporal graph foundation modeling.

Abstract

Temporal Graph Neural Networks (TGNNs) have gained growing attention for modeling and predicting structures in temporal graphs. However, existing TGNNs primarily focus on pairwise interactions while overlooking higher-order structures that are integral to link formation and evolution in real-world temporal graphs. Meanwhile, these models often suffer from efficiency bottlenecks, further limiting their expressive power. To tackle these challenges, we propose a Higher-order structure Temporal Graph Neural Network, which incorporates hypergraph representations into temporal graph learning. In particular, we develop an algorithm to identify the underlying higher-order structures, enhancing the model's ability to capture the group interactions. Furthermore, by aggregating multiple edge features into hyperedge representations, HTGN effectively reduces memory cost during training. We theoretically demonstrate the enhanced expressiveness of our approach and validate its effectiveness and efficiency through extensive experiments on various real-world temporal graphs. Experimental results show that HTGN achieves superior performance on dynamic link prediction while reducing memory costs by up to 50\% compared to existing methods.

Figures (7)

• Figure 1: The transformation between heterogeneous bipartite graphs and hypergraphs.
• Figure 2: The pipeline of HTGN. Left: the higher-order structure memory module. It will be dynamically updated as the stream of interaction events is input into the model (a homogeneous temporal graph in this example). Middle: the hypergraph embedding module. It will compute the node embeddings with the hyperedge features. Right: the link predictor. It will predict whether two nodes will form a link at the given time according to their node embeddings.
• Figure 3: The efficiency of HTGN compared to the baselines. The x-axis indicates the running time of a single epoch, the y-axis indicates the model's GPU memory costs. Lower x-axis and y-axis metrics indicate better efficiency. HTGN exhibits considerable efficiency advantages against the baselines, especially in GPU memory costs.
• Figure 4: The change of MRR compared to original HTGN after removing/adding modules. HM indicates higher-order structure memory, HE indicates hypergraph embedding, and NCN indicates the common-neighbor information wang2023neural.
• Figure 5: Two temporal graphs which HTGN can distinguish but MP-TGN can not. The colors represent node features.

Theorems & Definitions (20)

• Definition 3.1: Temporal Graphs
• Definition 3.2: Dynamic Link Prediction
• Definition 3.3: Hypergraph
• Definition 3.4: Maximal Cliques
• Definition 4.1: Homogeneous temporal graphs
• Definition 4.2: Heterogeneous temporal graphs
• Definition 4.3: Hypergraph temporal stochastic block model
• Theorem 1
• Definition 5.1
• Definition 5.2