Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

State Space Models on Temporal Graphs: A First-Principles Study

Jintang Li, Ruofan Wu, Xinzhou Jin, Boqun Ma, Liang Chen, Zibin Zheng

TL;DR

This work undertaking a principled investigation that extends SSM theory to temporal graphs by integrating structural information into the online approximation objective via the adoption of a Laplacian regularization term demonstrates the effectiveness of the GraphSSM framework across various temporal graph benchmarks.

Abstract

Over the past few years, research on deep graph learning has shifted from static graphs to temporal graphs in response to real-world complex systems that exhibit dynamic behaviors. In practice, temporal graphs are formalized as an ordered sequence of static graph snapshots observed at discrete time points. Sequence models such as RNNs or Transformers have long been the predominant backbone networks for modeling such temporal graphs. Yet, despite the promising results, RNNs struggle with long-range dependencies, while transformers are burdened by quadratic computational complexity. Recently, state space models (SSMs), which are framed as discretized representations of an underlying continuous-time linear dynamical system, have garnered substantial attention and achieved breakthrough advancements in independent sequence modeling. In this work, we undertake a principled investigation that extends SSM theory to temporal graphs by integrating structural information into the online approximation objective via the adoption of a Laplacian regularization term. The emergent continuous-time system introduces novel algorithmic challenges, thereby necessitating our development of GraphSSM, a graph state space model for modeling the dynamics of temporal graphs. Extensive experimental results demonstrate the effectiveness of our GraphSSM framework across various temporal graph benchmarks.

State Space Models on Temporal Graphs: A First-Principles Study

TL;DR

This work undertaking a principled investigation that extends SSM theory to temporal graphs by integrating structural information into the online approximation objective via the adoption of a Laplacian regularization term demonstrates the effectiveness of the GraphSSM framework across various temporal graph benchmarks.

Abstract

Over the past few years, research on deep graph learning has shifted from static graphs to temporal graphs in response to real-world complex systems that exhibit dynamic behaviors. In practice, temporal graphs are formalized as an ordered sequence of static graph snapshots observed at discrete time points. Sequence models such as RNNs or Transformers have long been the predominant backbone networks for modeling such temporal graphs. Yet, despite the promising results, RNNs struggle with long-range dependencies, while transformers are burdened by quadratic computational complexity. Recently, state space models (SSMs), which are framed as discretized representations of an underlying continuous-time linear dynamical system, have garnered substantial attention and achieved breakthrough advancements in independent sequence modeling. In this work, we undertake a principled investigation that extends SSM theory to temporal graphs by integrating structural information into the online approximation objective via the adoption of a Laplacian regularization term. The emergent continuous-time system introduces novel algorithmic challenges, thereby necessitating our development of GraphSSM, a graph state space model for modeling the dynamics of temporal graphs. Extensive experimental results demonstrate the effectiveness of our GraphSSM framework across various temporal graph benchmarks.
Paper Structure (47 sections, 2 theorems, 42 equations, 3 figures, 6 tables, 3 algorithms)

This paper contains 47 sections, 2 theorems, 42 equations, 3 figures, 6 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Temporal graph learning
  4. State space models
  5. Comparison.
  6. The GraphSSM framework
  7. GHiPPO: HiPPO on temporal graphs
  8. Unobserved graph mutations and mixed discretization
  9. The GraphSSM framework
  10. Experiments
  11. Experimental results
  12. Node classification performance.
  13. Scalability to large temporal graphs.
  14. SSM architectures.
  15. Mixing mechanism.
  16. ...and 32 more sections

Key Result

Theorem 1

Let $G$ evolve according to eqn: graph_evolve. Taking $\mu_t$ to be the scaled Legendre measure (LegS) with $\mu_t = \frac{1}{t}\mathbb{I}_{[0, t]}$ where $\mathbb{I}_{[0, t]}$ stands for the indicator function of the interval $[0, t]$, the evolution of the outputs of GHiPPO operator is characterize where $A \in \mathbb{R}^{N \times N}$ and $B \in \mathbb{R}^{N \times 1}$ takes the same form as in

Figures (3)

  • Figure 1: GraphSSM framework.
  • Figure 2: Illustrative example of the unobserved graph mutation issue. In this example, the underlying graph is observed at time points $\tau_1, \tau_2, \tau_3$ with two unobserved mutations between $[\tau_1, \tau_2)$ and one between $[\tau_2, \tau_3)$. These unobserved mutations result in temporal dynamics that are inconsistent across the observed intervals, thereby complicating direct applications of ODE discretization methods such as the Euler method or the zero-order hold (ZOH) method.
  • Figure 3: Comparison of GraphSSM with different initialization strategies.

Theorems & Definitions (6)

  • Definition 1: GHiPPO
  • Theorem 1
  • Theorem 2: Oracle discretization of \ref{['eqn: ghippo']}
  • Remark 1: Choice of mixing mechanisms
  • proof : Proof of theorem \ref{['thm: ghippo']}
  • proof : Proof of theorem \ref{['thm: zoh']}