Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Discrete-state Continuous-time Diffusion for Graph Generation

Zhe Xu, Ruizhong Qiu, Yuzhong Chen, Huiyuan Chen, Xiran Fan, Menghai Pan, Zhichen Zeng, Mahashweta Das, Hanghang Tong

TL;DR

The rationale of such a formulation is to preserve the discrete nature of graph-structured data and meanwhile provide flexible sampling trade-offs between sample quality and efficiency and the proposed model shows competitive empirical performance against state-of-the-art graph generation solutions on various benchmarks.

Abstract

Graph is a prevalent discrete data structure, whose generation has wide applications such as drug discovery and circuit design. Diffusion generative models, as an emerging research focus, have been applied to graph generation tasks. Overall, according to the space of states and time steps, diffusion generative models can be categorized into discrete-/continuous-state discrete-/continuous-time fashions. In this paper, we formulate the graph diffusion generation in a discrete-state continuous-time setting, which has never been studied in previous graph diffusion models. The rationale of such a formulation is to preserve the discrete nature of graph-structured data and meanwhile provide flexible sampling trade-offs between sample quality and efficiency. Analysis shows that our training objective is closely related to generation quality, and our proposed generation framework enjoys ideal invariant/equivariant properties concerning the permutation of node ordering. Our proposed model shows competitive empirical performance against state-of-the-art graph generation solutions on various benchmarks and, at the same time, can flexibly trade off the generation quality and efficiency in the sampling phase.

Discrete-state Continuous-time Diffusion for Graph Generation

TL;DR

The rationale of such a formulation is to preserve the discrete nature of graph-structured data and meanwhile provide flexible sampling trade-offs between sample quality and efficiency and the proposed model shows competitive empirical performance against state-of-the-art graph generation solutions on various benchmarks.

Abstract

Graph is a prevalent discrete data structure, whose generation has wide applications such as drug discovery and circuit design. Diffusion generative models, as an emerging research focus, have been applied to graph generation tasks. Overall, according to the space of states and time steps, diffusion generative models can be categorized into discrete-/continuous-state discrete-/continuous-time fashions. In this paper, we formulate the graph diffusion generation in a discrete-state continuous-time setting, which has never been studied in previous graph diffusion models. The rationale of such a formulation is to preserve the discrete nature of graph-structured data and meanwhile provide flexible sampling trade-offs between sample quality and efficiency. Analysis shows that our training objective is closely related to generation quality, and our proposed generation framework enjoys ideal invariant/equivariant properties concerning the permutation of node ordering. Our proposed model shows competitive empirical performance against state-of-the-art graph generation solutions on various benchmarks and, at the same time, can flexibly trade off the generation quality and efficiency in the sampling phase.
Paper Structure (52 sections, 9 theorems, 38 equations, 5 figures, 9 tables, 2 algorithms)

This paper contains 52 sections, 9 theorems, 38 equations, 5 figures, 9 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Discrete-State Continuous-time Diffusion Models
  4. Graph Generation and Notations
  5. Method
  6. Factorized Discrete Graph Diffusion Process
  7. Forward Process
  8. Parameterization and Optimization Objective
  9. Sampling Reverse Process
  10. Model Instantiation
  11. Permutation Equivariance and Invariance
  12. Experiments
  13. Plain Graph Generation
  14. Molecule Graph Generation
  15. Efficiency Study
  16. ...and 37 more sections

Key Result

Proposition 3.2

The forward processes for nodes and edges converge to uniform distributions if $\mathbf{R}_f=\mathbf{1}\mathbf{1}^{\top}-b\mathbf{I}$ and $\mathbf{R}_e=\mathbf{1}\mathbf{1}^{\top}-(a+1)\mathbf{I}$; they converge to marginal distributions $\mathbf{m}_f$ and $\mathbf{m}_e$ if $\mathbf{R}_f=\mathbf{1}\

Figures (5)

  • Figure 1: A taxonomy of graph diffusion models.
  • Figure 2: An overview of DisCo. A transition can happen at any time in $[0, T]$.
  • Figure 3: Training loss of DisCo on different datasets and backbone models.
  • Figure 4: Generated graphs.
  • Figure 5: Generation trajectory of SBM graphs with different sizes. Every row is the generation trajectory of one graph from time $t=T$ (left) to $t=0$ (right) with equal time intervals.

Theorems & Definitions (19)

  • Remark 3.1
  • Proposition 3.2
  • Theorem 3.3: Approximation error
  • Remark 3.4
  • Lemma 3.5: Permutation-equivariant layer
  • Lemma 3.6: Permutation-invariant rate matrices
  • Lemma 3.7: Permutation-invariant transition probability
  • Theorem 3.8: Permutation-invariant sampling probability
  • Theorem 3.9: Permutation-invariant training loss
  • Proposition A.1: Factorization of the rate matrix, Proposition 3 from DBLP:conf/nips/CampbellBBRDD22
  • ...and 9 more