Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DURENDAL: Graph deep learning framework for temporal heterogeneous networks

Manuel Dileo, Matteo Zignani, Sabrina Gaito

TL;DR

DURENDAL introduces a general framework to extend heterogeneous GNNs to temporal heterogeneous networks by using hierarchical node states and flexible semantic aggregation. It delivers two embedding-update schemes—Update-Then-Aggregate and Aggregate-Then-Update—and supports live-update training to reflect the evolving nature of THNs. The approach is demonstrated on four high-resolution THN datasets, showing improved performance over dynamic baselines on most monorelational and several multirelational tasks, and it highlights when each update scheme provides the best trade-off between learning power and memory efficiency. This work also expands THN benchmarks with new datasets and offers practical guidance for repurposing static THN models to dynamic contexts, with implications for recommendation, knowledge graphs, and event prediction. The results underscore the value of flexible, task-agnostic THN modeling in real-world, evolving relational data scenarios.

Abstract

Temporal heterogeneous networks (THNs) are evolving networks that characterize many real-world applications such as citation and events networks, recommender systems, and knowledge graphs. Although different Graph Neural Networks (GNNs) have been successfully applied to dynamic graphs, most of them only support homogeneous graphs or suffer from model design heavily influenced by specific THNs prediction tasks. Furthermore, there is a lack of temporal heterogeneous networked data in current standard graph benchmark datasets. Hence, in this work, we propose DURENDAL, a graph deep learning framework for THNs. DURENDAL can help to easily repurpose any heterogeneous graph learning model to evolving networks by combining design principles from snapshot-based and multirelational message-passing graph learning models. We introduce two different schemes to update embedding representations for THNs, discussing the strengths and weaknesses of both strategies. We also extend the set of benchmarks for TNHs by introducing two novel high-resolution temporal heterogeneous graph datasets derived from an emerging Web3 platform and a well-established e-commerce website. Overall, we conducted the experimental evaluation of the framework over four temporal heterogeneous network datasets on future link prediction tasks in an evaluation setting that takes into account the evolving nature of the data. Experiments show the prediction power of DURENDAL compared to current solutions for evolving and dynamic graphs, and the effectiveness of its model design.

DURENDAL: Graph deep learning framework for temporal heterogeneous networks

TL;DR

DURENDAL introduces a general framework to extend heterogeneous GNNs to temporal heterogeneous networks by using hierarchical node states and flexible semantic aggregation. It delivers two embedding-update schemes—Update-Then-Aggregate and Aggregate-Then-Update—and supports live-update training to reflect the evolving nature of THNs. The approach is demonstrated on four high-resolution THN datasets, showing improved performance over dynamic baselines on most monorelational and several multirelational tasks, and it highlights when each update scheme provides the best trade-off between learning power and memory efficiency. This work also expands THN benchmarks with new datasets and offers practical guidance for repurposing static THN models to dynamic contexts, with implications for recommendation, knowledge graphs, and event prediction. The results underscore the value of flexible, task-agnostic THN modeling in real-world, evolving relational data scenarios.

Abstract

Temporal heterogeneous networks (THNs) are evolving networks that characterize many real-world applications such as citation and events networks, recommender systems, and knowledge graphs. Although different Graph Neural Networks (GNNs) have been successfully applied to dynamic graphs, most of them only support homogeneous graphs or suffer from model design heavily influenced by specific THNs prediction tasks. Furthermore, there is a lack of temporal heterogeneous networked data in current standard graph benchmark datasets. Hence, in this work, we propose DURENDAL, a graph deep learning framework for THNs. DURENDAL can help to easily repurpose any heterogeneous graph learning model to evolving networks by combining design principles from snapshot-based and multirelational message-passing graph learning models. We introduce two different schemes to update embedding representations for THNs, discussing the strengths and weaknesses of both strategies. We also extend the set of benchmarks for TNHs by introducing two novel high-resolution temporal heterogeneous graph datasets derived from an emerging Web3 platform and a well-established e-commerce website. Overall, we conducted the experimental evaluation of the framework over four temporal heterogeneous network datasets on future link prediction tasks in an evaluation setting that takes into account the evolving nature of the data. Experiments show the prediction power of DURENDAL compared to current solutions for evolving and dynamic graphs, and the effectiveness of its model design.
Paper Structure (7 sections, 5 equations, 4 figures, 7 tables)

This paper contains 7 sections, 5 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. The proposed framework: DURENDAL
  4. Temporal heterogeneous networks dataset
  5. Experimental evaluation
  6. Conclusion
  7. Appendix

Figures (4)

  • Figure 1: DURENDAL model design. (a) Scheme of the computation beyond a heterogeneous GNN layer. (b) Compact representation of the (a) scheme within the GRAPHEDM paradigm chami2022machine. (c) DURENDAL framework with the Update-Then-Aggregate scheme: the orange layer (temporal layer) updates over time the hierarchical node state of each relation type (returned by the first two layers in (b)), then the aggregation scheme (yellow) is run on top the temporal layer. In the Aggregate-Then-Update scheme the temporal layer and the aggregation scheme are swapped.
  • Figure 2: (a) AUPRC distributions of DURENDAL repurposed RGCN, HAN, and HGT on future link prediction task on SteemitTH, using ConcatMLP or a GRU cell as embedding update module (experiments with 10 random seeds). Attention-based aggregation schemes and ConcatMLP update modules are desirable for GNN architectural design. (b) Results snapshot-by-snapshots of DURENDAL models for "follow" link prediction on SteemitTH using different update schemes. UTA outperforms the other update schemes but ATU is still a profitable choice for learning on multiple snapshots.
  • Figure 3: Execution time of DURENDAL on ICEWS18 and GDELT18 with different embedding size.
  • Figure 4: DURENDAL framework with UTA schema and customizable encoder function for node embeddings. ENC could be any Graph Representation Learning grl-hamilton encoder function for node embeddings, such as GNNs, random-walk, or tensor factorization-based methods.