Multi-perspective Feedback-attention Coupling Model for Continuous-time Dynamic Graphs

TL;DR

MPFA addresses limitations of prior continuous-time dynamic-graph methods by learning from both evolving and original perspectives, enabling efficient long-term dependency capture with few temporal neighbors. The evolving perspective applies temporal self-attention to aggregate current-state information, while the original perspective uses growth-based feedback attention; the two views are coupled to produce robust node embeddings. Across eight datasets, MPFA achieves state-of-the-art performance on dynamic link prediction and dynamic node classification, as shown by extensive ablations and efficiency analyses. This dual-perspective framework offers a principled and scalable approach for representing continuously evolving graphs with broad applicability to social networks, recommendations, and dynamic knowledge graphs.

Abstract

Recently, representation learning over graph networks has gained popularity, with various models showing promising results. Despite this, several challenges persist: 1) most methods are designed for static or discrete-time dynamic graphs; 2) existing continuous-time dynamic graph algorithms focus on a single evolving perspective; and 3) many continuous-time dynamic graph approaches necessitate numerous temporal neighbors to capture long-term dependencies. In response, this paper introduces the Multi-Perspective Feedback-Attention Coupling (MPFA) model. MPFA incorporates information from both evolving and raw perspectives, efficiently learning the interleaved dynamics of observed processes. The evolving perspective employs temporal self-attention to distinguish continuously evolving temporal neighbors for information aggregation. Through dynamic updates, this perspective can capture long-term dependencies using a small number of temporal neighbors. Meanwhile, the raw perspective utilizes a feedback attention module with growth characteristic coefficients to aggregate raw neighborhood information. Experimental results on a self-organizing dataset and seven public datasets validate the efficacy and competitiveness of our proposed model.

Figures (9)

• Figure 1: A simple example illustrates a continuous-time dynamic graph with multiple complex dynamics, where $t_0$-$t_3$ represent timestamps. The evolution of the graph from $t_0$ to $t_3$ captures the entire process, revealing that new interactions between nodes are established at each timestamp. At $t_1$, a new node joins the network and interacts with another node (red edge). At $t_2$, co-occurrence events occur (green edges), and at the final timestamp, the connection between nodes 1 and 6 is re-established (blue edge). Conversely, a discrete-time dynamic graph can be formed by extracting snapshots (e.g., snapshots $t_0$ and $t_2$) from the temporal network. If only the final state of the graph evolution is considered, a static graph can be obtained.
• Figure 2: The Original and current network states from two perspectives. The model learns embedding representations for nodes $v_i$ and $v_6$ at time $t$ and predicts the occurrence of the event $x(t)=(v_i, v_6, e_{1,6}(t))$. For node $v_i$ (and similarly for node $v_6$), the orange dashed arrow represents feedback from the current state of node $v_i$ to the original state of the interacting node. The purple dashed arrow represents feedback from the current states of node $v_i$'s neighbors to their original states. The blue and red solid arrows show the influence of original and current state information on the embedding representation of node $v_i$, respectively.
• Figure 3: The overall architecture of MPFA. $t_{p_1}-t_{p_6}$ denote the interaction time before time $t$. The evolving perspective (bottom) aggregates the current state information of historical events by using a temporal self-attention module; For the original perspective (top), the feedback attentions with growth characteristics calculated by the feedback coefficient module are utilized to aggregate the original state information; In the above two processes, the original and evolving information preserved modules play a critical role. Then, the acquired evolving and original features are coupled to teach each other by the attention coupling module for the final temporal graph embedding. After completing the learning of the two perspectives (gray arrow process), MPFA performs the updating of the ESP-OSP (red arrow process).
• Figure 4: Results of ablation experiments. The results of the MOOC ablation experiments are denoted by (a) and (b), and for the USLegis dataset by (c) and (d). W/O RP: without Raw Perspective; W/O EP: without Evolving Perspective; W/O RED: Raw and Evolving perspectives without Dynamic update; W/O ED: Evolving perspective without Dynamic update.
• Figure 5: Long-term dependency effects of six models on MOOC and USLeigs datasets under different numbers of neighbors.