Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DG-Mamba: Robust and Efficient Dynamic Graph Structure Learning with Selective State Space Models

Haonan Yuan, Qingyun Sun, Zhaonan Wang, Xingcheng Fu, Cheng Ji, Yongjian Wang, Bo Jin, Jianxin Li

TL;DR

DG-Mamba tackles robustness and efficiency in dynamic graph structure learning by combining a kernelized dynamic message-passing operator with a Selective State Space Model and a self-supervised Principle of Relevant Information. The kernelization reduces spatial-temporal quadratic complexity to linear, while the State Space Model captures long-range dependencies across snapshots through discretized inter-graph structures. PRI regularization promotes informative yet minimal structural representations, improving robustness against adversarial attacks. Extensive experiments on real-world and synthetic datasets demonstrate that DG-Mamba outperforms 12 baselines in robustness and efficiency, especially under adversarial perturbations.

Abstract

Dynamic graphs exhibit intertwined spatio-temporal evolutionary patterns, widely existing in the real world. Nevertheless, the structure incompleteness, noise, and redundancy result in poor robustness for Dynamic Graph Neural Networks (DGNNs). Dynamic Graph Structure Learning (DGSL) offers a promising way to optimize graph structures. However, aside from encountering unacceptable quadratic complexity, it overly relies on heuristic priors, making it hard to discover underlying predictive patterns. How to efficiently refine the dynamic structures, capture intrinsic dependencies, and learn robust representations, remains under-explored. In this work, we propose the novel DG-Mamba, a robust and efficient Dynamic Graph structure learning framework with the Selective State Space Models (Mamba). To accelerate the spatio-temporal structure learning, we propose a kernelized dynamic message-passing operator that reduces the quadratic time complexity to linear. To capture global intrinsic dynamics, we establish the dynamic graph as a self-contained system with State Space Model. By discretizing the system states with the cross-snapshot graph adjacency, we enable the long-distance dependencies capturing with the selective snapshot scan. To endow learned dynamic structures more expressive with informativeness, we propose the self-supervised Principle of Relevant Information for DGSL to regularize the most relevant yet least redundant information, enhancing global robustness. Extensive experiments demonstrate the superiority of the robustness and efficiency of our DG-Mamba compared with the state-of-the-art baselines against adversarial attacks.

DG-Mamba: Robust and Efficient Dynamic Graph Structure Learning with Selective State Space Models

TL;DR

DG-Mamba tackles robustness and efficiency in dynamic graph structure learning by combining a kernelized dynamic message-passing operator with a Selective State Space Model and a self-supervised Principle of Relevant Information. The kernelization reduces spatial-temporal quadratic complexity to linear, while the State Space Model captures long-range dependencies across snapshots through discretized inter-graph structures. PRI regularization promotes informative yet minimal structural representations, improving robustness against adversarial attacks. Extensive experiments on real-world and synthetic datasets demonstrate that DG-Mamba outperforms 12 baselines in robustness and efficiency, especially under adversarial perturbations.

Abstract

Dynamic graphs exhibit intertwined spatio-temporal evolutionary patterns, widely existing in the real world. Nevertheless, the structure incompleteness, noise, and redundancy result in poor robustness for Dynamic Graph Neural Networks (DGNNs). Dynamic Graph Structure Learning (DGSL) offers a promising way to optimize graph structures. However, aside from encountering unacceptable quadratic complexity, it overly relies on heuristic priors, making it hard to discover underlying predictive patterns. How to efficiently refine the dynamic structures, capture intrinsic dependencies, and learn robust representations, remains under-explored. In this work, we propose the novel DG-Mamba, a robust and efficient Dynamic Graph structure learning framework with the Selective State Space Models (Mamba). To accelerate the spatio-temporal structure learning, we propose a kernelized dynamic message-passing operator that reduces the quadratic time complexity to linear. To capture global intrinsic dynamics, we establish the dynamic graph as a self-contained system with State Space Model. By discretizing the system states with the cross-snapshot graph adjacency, we enable the long-distance dependencies capturing with the selective snapshot scan. To endow learned dynamic structures more expressive with informativeness, we propose the self-supervised Principle of Relevant Information for DGSL to regularize the most relevant yet least redundant information, enhancing global robustness. Extensive experiments demonstrate the superiority of the robustness and efficiency of our DG-Mamba compared with the state-of-the-art baselines against adversarial attacks.

Paper Structure

This paper contains 72 sections, 3 theorems, 40 equations, 16 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Robust Dynamic Graph Learning
  4. Dynamic Graph Structure Learning
  5. Graph Modeling with State Space Models
  6. Preliminary
  7. DG-Mamba: Robust and Efficient Dynamic Graph Structure Learning
  8. Kernelized Message-Passing for Efficient Dynamic Graph Structure Learning
  9. Long-range Dependencies Selective Modeling
  10. Robust Relevant Structure Regularizing
  11. Optimization and Complexity Analysis
  12. Computational Complexity.
  13. Experiment
  14. Experimental Settings
  15. Dynamic Graph Datasets.
  16. ...and 57 more sections

Key Result

Proposition 1

Suppose $\|\mathbf{W}\mathbf{z}_u^t\|_2$ and $\|\mathbf{W}\mathbf{z}_v^t\|_2$ are bounded by some $r$, then for any positive $\epsilon$, the approximation error $\Delta = |\phi(\mathbf{W}\mathbf{z}_u^{t}/\sqrt{\tau})^\top\phi(\mathbf{W}\mathbf{z}_v^{t}/\sqrt{\tau}) - k(\mathbf{W}\mathbf{z}_u^{t}/\sq where $m$ denotes the projection dimension in the Reproducing Kernel Hilbert Space $\mathcal{H}$.

Figures (16)

  • Figure 1: A general paradigm of DGSL.
  • Figure 2: The framework of DG-Mamba. (a) Kernelized message-passing mechanism learns both intra- and inter-graph weights with linear time complexity. (b) Long-range dependencies are strengthened by selective modeling with parameters discretized by learned inter-graph structures. (c) PRI for DGSL is proposed to guarantee robustness against noise and adversarial attacks.
  • Figure 3: Node scaling efficiency evaluation on Yelp.
  • Figure 4: Sequence scaling efficiency evaluation on Yelp.
  • Figure 5: Ablation study.
  • ...and 11 more figures

Theorems & Definitions (6)

  • Definition 1: PRI for DGSL
  • Proposition 1: Gumbel-Softmax Kernel Approximation Error Bound
  • proof
  • Lemma 1: choromanski2020rethinkingwu2022nodeformer
  • Proposition 2: Edge-Level Constraints Loss Equivalance
  • proof