DyGSSM: Multi-view Dynamic Graph Embeddings with State Space Model Gradient Update

TL;DR

DyGSSM tackles the challenge of learning robust representations for dynamic graphs by jointly modeling local and global structure within each snapshot and by updating model parameters through a HiPPO-based state-space memory that preserves long-term temporal dependencies. The method uses a lightweight two-stream embedding scheme (GCN-based local and biased random-walk-based global), fused via cross-attention, and updates parameters with a compressed HiPPO state per tensor to avoid heavy hyperparameter tuning. Empirical results across discrete and continuous-time benchmarks show DyGSSM achieving strong, scalable performance with fewer parameters than many baselines, aided by a caching mechanism that speeds up random walks. The work demonstrates the value of integrating state-space memory with meta-learning-inspired parameter updates for dynamic graphs, enabling efficient, long-horizon reasoning in streaming settings.

Abstract

Most of the dynamic graph representation learning methods involve dividing a dynamic graph into discrete snapshots to capture the evolving behavior of nodes over time. Existing methods primarily capture only local or global structures of each node within a snapshot using message-passing and random walk-based methods. Then, they utilize sequence-based models (e.g., transformers) to encode the temporal evolution of node embeddings, and meta-learning techniques to update the model parameters. However, these approaches have two limitations. First, they neglect the extraction of global and local information simultaneously in each snapshot. Second, they fail to consider the model's performance in the current snapshot during parameter updates, resulting in a lack of temporal dependency management. Recently, HiPPO (High-order Polynomial Projection Operators) algorithm has gained attention for their ability to optimize and preserve sequence history in State Space Model (SSM). To address the aforementioned limitations in dynamic graph representation learning, we propose a novel method called Multi-view Dynamic Graph Embeddings with State Space Model Gradient Update (DyGSSM). Our approach combines Graph Convolution Networks (GCN) for local feature extraction and random walk with Gated Recurrent Unit (GRU) for global feature extraction in each snapshot. We then integrate the local and global features using a cross-attention mechanism. Additionally, we incorporate an SSM based on HiPPO algorithm to account for long-term dependencies when updating model parameters, ensuring that model performance in each snapshot informs subsequent updates. Experiments on five public datasets show that our method outperforms existing baseline and state-of-the-art (SOTA) methods in 17 out of 20 cases.

Figures (6)

• Figure 1: DyGSSM architecture.
• Figure 2: The figure compares how ROLAND, WinGNN, and DyGSSM update their model parameters. (a) ROLAND updates model parameters between consecutive time steps using fixed meta-learning weights. (b) WinGNN updates parameters between time steps with a fixed learning rate. Instead of relying on explicit time encoding, it uses a window-based gradient aggregation mechanism. (c) DyGSSM utilizes the HiPPO-based algorithm to update model parameters without a need to specify a window size.
• Figure 3: Global neighborhood computation time using RW on DBLP, Reddit-Title, and StackOverflow datasets after applying the caching mechanism. The computation cost per snapshot is initially high, but it significantly decreases when the caching mechanism is used.
• Figure 4: Model parameter size comparison discrete-time dynamic graphs. Each bar represents a model and its number of learnable parameters in millions (M) or thousands (K). DyGSSM consistently has one of the smallest parameter sizes, typically ranging from 50K to 92K. Despite integrating GCN, Conv1D, and light attention, our model remains lightweight and highly scalable.
• Figure 5: Model parameter size comparison (continuous-time dynamic graph). Each bar represents a model and its number of learnable parameters in millions (M) or thousands (K). DyGSSM has the smallest parameter size.