Towards Dynamic Graph Neural Networks with Provably High-Order Expressive Power

TL;DR

This work proposes Dynamic Graph Neural Network with High-order expressive power (HopeDGN), which updates the representation of central node pair by aggregating the interaction history with neighboring node pairs, and theoretical results demonstrate that HopeDGN can achieve expressive power equivalent to the 2-DWL test.

Abstract

Dynamic Graph Neural Networks (DyGNNs) have garnered increasing research attention for learning representations on evolving graphs. Despite their effectiveness, the limited expressive power of existing DyGNNs hinders them from capturing important evolving patterns of dynamic graphs. Although some works attempt to enhance expressive capability with heuristic features, there remains a lack of DyGNN frameworks with provable and quantifiable high-order expressive power. To address this research gap, we firstly propose the k-dimensional Dynamic WL tests (k-DWL) as the referencing algorithms to quantify the expressive power of DyGNNs. We demonstrate that the expressive power of existing DyGNNs is upper bounded by the 1-DWL test. To enhance the expressive power, we propose Dynamic Graph Neural Network with High-order expressive power (HopeDGN), which updates the representation of central node pair by aggregating the interaction history with neighboring node pairs. Our theoretical results demonstrate that HopeDGN can achieve expressive power equivalent to the 2-DWL test. We then present a Transformer-based implementation for the local variant of HopeDGN. Experimental results show that HopeDGN achieved performance improvements of up to 3.12%, demonstrating the effectiveness of HopeDGN.

Figures (5)

• Figure 1: An example of limited expressive power of DyGNNs. Suppose the model is distinguishing node pairs $(A,C)$ and $(A,D)$ at time $t_4$. Because $A$ and $C$ have historical interaction with $B$ while $A$ and $D$ do not have common historical interacted nodes, $(A,C)$ and $(A,D)$ are not isomorphic at time $t_4$. Since nodes $C$ and $D$ are isomorphic on the historical interaction graph before $t_4$, DyGNNs will output the same embeddings for $(C,t_4)$ and $(D,t_4)$. Thus, DyGNNs fails to distinguish $(A,C,t_4)$ and $(A,D,t_4)$. Conversely, HopeDGN will notice that node $B$ interacts with $A$ and $C$ at $t_1$ and $t_2$ respectively, thus being capable of distinguish these node pairs.
• Figure 2: Ablation studies on the components of HopeDGN.
• Figure 3: An example of Proposition \ref{['proposition: bite']}. Suppose the raw node feature are same for all nodes, and the current time is $t_5$. The model is required to distinguish two node pairs $(a,c)$ in (a) and $(a,g)$ in (b) at time $t_5$.
• Figure 4: Parameter sensitivity of HopeDGN.
• Figure 5: Left: Efficiency-performance comparison of different models on MOOC dataset. The X-axis is the training time per epoch (seconds). The Y-axis is the inductive AP value. 'HopeDGN-$n$' denotes HopeDGN with input neighbor length of $n$. Right: Training time of HopeDGN with various neighbor length on MOOC dataset.

Theorems & Definitions (15)

• Proposition 1
• Proposition 2
• Proposition 3
• Proposition 4
• Proposition 5
• Proposition 6
• proof
• Proposition 7
• proof
• Proposition 8