Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast Graph Generation via Spectral Diffusion

Tianze Luo, Zhanfeng Mo, Sinno Jialin Pan

TL;DR

Graph diffusion for graphs faces challenges when diffusion acts on the full adjacency space, which can blur topology. The paper introduces Graph Spectral Diffusion Model (GSDM), which performs diffusion on the graph spectrum (eigenvalues) while keeping eigenvectors fixed, yielding low-rank diffusion and improved efficiency. Theoretical analysis shows a sharper reconstruction bound for spectral diffusion and closed-form characterizations of the diffusion process in the spectral domain. Empirically, GSDM achieves state-of-the-art generation quality on generic graphs and molecule datasets (QM9, ZINC) with significantly faster inference, and the alpha-quantile variant further accelerates computation with minimal performance loss.

Abstract

Generating graph-structured data is a challenging problem, which requires learning the underlying distribution of graphs. Various models such as graph VAE, graph GANs, and graph diffusion models have been proposed to generate meaningful and reliable graphs, among which the diffusion models have achieved state-of-the-art performance. In this paper, we argue that running full-rank diffusion SDEs on the whole graph adjacency matrix space hinders diffusion models from learning graph topology generation, and hence significantly deteriorates the quality of generated graph data. To address this limitation, we propose an efficient yet effective Graph Spectral Diffusion Model (GSDM), which is driven by low-rank diffusion SDEs on the graph spectrum space. Our spectral diffusion model is further proven to enjoy a substantially stronger theoretical guarantee than standard diffusion models. Extensive experiments across various datasets demonstrate that, our proposed GSDM turns out to be the SOTA model, by exhibiting both significantly higher generation quality and much less computational consumption than the baselines.

Fast Graph Generation via Spectral Diffusion

TL;DR

Graph diffusion for graphs faces challenges when diffusion acts on the full adjacency space, which can blur topology. The paper introduces Graph Spectral Diffusion Model (GSDM), which performs diffusion on the graph spectrum (eigenvalues) while keeping eigenvectors fixed, yielding low-rank diffusion and improved efficiency. Theoretical analysis shows a sharper reconstruction bound for spectral diffusion and closed-form characterizations of the diffusion process in the spectral domain. Empirically, GSDM achieves state-of-the-art generation quality on generic graphs and molecule datasets (QM9, ZINC) with significantly faster inference, and the alpha-quantile variant further accelerates computation with minimal performance loss.

Abstract

Generating graph-structured data is a challenging problem, which requires learning the underlying distribution of graphs. Various models such as graph VAE, graph GANs, and graph diffusion models have been proposed to generate meaningful and reliable graphs, among which the diffusion models have achieved state-of-the-art performance. In this paper, we argue that running full-rank diffusion SDEs on the whole graph adjacency matrix space hinders diffusion models from learning graph topology generation, and hence significantly deteriorates the quality of generated graph data. To address this limitation, we propose an efficient yet effective Graph Spectral Diffusion Model (GSDM), which is driven by low-rank diffusion SDEs on the graph spectrum space. Our spectral diffusion model is further proven to enjoy a substantially stronger theoretical guarantee than standard diffusion models. Extensive experiments across various datasets demonstrate that, our proposed GSDM turns out to be the SOTA model, by exhibiting both significantly higher generation quality and much less computational consumption than the baselines.
Paper Structure (22 sections, 7 theorems, 39 equations, 12 figures, 3 tables, 2 algorithms)

This paper contains 22 sections, 7 theorems, 39 equations, 12 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Notations
  5. Score-based Generative Diffusion Model
  6. Methodology
  7. Standard Graph Diffusion Model
  8. Graph Spectral Diffusion Model
  9. Theoretical Analysis
  10. Experiments
  11. Generic Graph Generation
  12. Molecules Generation
  13. Ablation Studies
  14. Conclusion
  15. Proof Details
  16. ...and 7 more sections

Key Result

Lemma 3.1

The Forward Diffusion refers to the following SDE where $\mathbf{f}(\cdot, t): \mathbb{R}^d \mapsto \mathbb{R}^d$ is the drift function, $\sigma_{t}: [0,1] \mapsto \mathbb{R}$ be a scalar diffusion function. Let $p_t(\cdot)$ be the probability density function of $\mathbf{z}_t$, then the Reversed Time SDE is given by where $\mathrm{d}\bar{t}=-\mathrm{d}t$ is the negative infinitesimal time step,

Figures (12)

  • Figure 1: Illustation of the difference between applying the conventional SDE diffusion process on images (shown in (a)) and on graphs (shown in (b)).
  • Figure 2: Non-cherry-picked random samples from the testing set as well as samples generated by GSDM (ours) and GDSS jo2022score, on Grid (top row) and Community-small (bottom row) datasets. For GDSS, we use the authors' released code and checkpoints to generate the samples.
  • Figure 3: GSDM enjoys a significantly faster convergence rate than GDSS. On the community-small dataset, our GSDM reaches the SOTA performance within 100 training epochs.
  • Figure 4: Ablation studies on different diffusion step numbers.
  • Figure 5: Ablation studies on diffusion schedules. The y-axis denotes the average performance score ($\downarrow$) and each column denotes a diffusion schedule configuration. Each box plot summarizes the results of 5 random trials.
  • ...and 7 more figures

Theorems & Definitions (16)

  • Lemma 3.1: Forward Diffusion and Reversed Time SDE reversed_time_sde_anderson
  • Definition 1: Graph Diffusion SDEs with Disentangled Drift
  • Corollary 1: Reversed Time SDEs for Graph Diffusion
  • Definition 2: Graph Spectral Diffusion SDEs
  • Corollary 2: Reversed Time Spectral Diffusion SDEs
  • Proposition 1: Spectral Diffusion SDEs on Adjacency Matrix
  • Remark 1
  • Proposition 2: Reconstruction Bounds for Adjacency Matrix Generation
  • Remark 2
  • Definition 3: $\beta$-smooth
  • ...and 6 more