Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cometh: A continuous-time discrete-state graph diffusion model

Antoine Siraudin, Fragkiskos D. Malliaros, Christopher Morris

TL;DR

Cometh is proposed, a continuous-time discrete-state graph diffusion model, tailored to the specificities of graph data, which shows that integrating continuous time leads to significant improvements across various metrics over state-of-the-art discrete-state diffusion models on a large set of molecular and non-molecular benchmark datasets.

Abstract

Discrete-state denoising diffusion models led to state-of-the-art performance in graph generation, especially in the molecular domain. Recently, they have been transposed to continuous time, allowing more flexibility in the reverse process and a better trade-off between sampling efficiency and quality. Here, to leverage the benefits of both approaches, we propose Cometh, a continuous-time discrete-state graph diffusion model, tailored to the specificities of graph data. In addition, we also successfully replaced the set of structural encodings previously used in the discrete graph diffusion model with a single random-walk-based encoding, providing a simple and principled way to boost the model's expressive power. Empirically, we show that integrating continuous time leads to significant improvements across various metrics over state-of-the-art discrete-state diffusion models on a large set of molecular and non-molecular benchmark datasets. In terms of VUN samples, Cometh obtains a near-perfect performance of 99.5% on the planar graph dataset and outperforms DiGress by 12.6% on the large GuacaMol dataset.

Cometh: A continuous-time discrete-state graph diffusion model

TL;DR

Cometh is proposed, a continuous-time discrete-state graph diffusion model, tailored to the specificities of graph data, which shows that integrating continuous time leads to significant improvements across various metrics over state-of-the-art discrete-state diffusion models on a large set of molecular and non-molecular benchmark datasets.

Abstract

Discrete-state denoising diffusion models led to state-of-the-art performance in graph generation, especially in the molecular domain. Recently, they have been transposed to continuous time, allowing more flexibility in the reverse process and a better trade-off between sampling efficiency and quality. Here, to leverage the benefits of both approaches, we propose Cometh, a continuous-time discrete-state graph diffusion model, tailored to the specificities of graph data. In addition, we also successfully replaced the set of structural encodings previously used in the discrete graph diffusion model with a single random-walk-based encoding, providing a simple and principled way to boost the model's expressive power. Empirically, we show that integrating continuous time leads to significant improvements across various metrics over state-of-the-art discrete-state diffusion models on a large set of molecular and non-molecular benchmark datasets. In terms of VUN samples, Cometh obtains a near-perfect performance of 99.5% on the planar graph dataset and outperforms DiGress by 12.6% on the large GuacaMol dataset.
Paper Structure (28 sections, 12 theorems, 40 equations, 8 figures, 9 tables, 2 algorithms)

This paper contains 28 sections, 12 theorems, 40 equations, 8 figures, 9 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. A Continuous-Time Discrete-State Graph Diffusion Model
  4. Experiments
  5. Synthetic graph generation: Planar and SBM
  6. Small-molecule generation: QM9
  7. Molecule generation on large datasets: Moses and GuacaMol
  8. Conditional generation
  9. Conclusion
  10. Theoretical details
  11. Additional background
  12. Noise schedule
  13. RRWP properties
  14. Equivariance properties
  15. Classifier-free guidance
  16. ...and 13 more sections

Key Result

Proposition 1

For a CTMC $(z^{(t)})_{t \in [0,1]}$ with rate matrix ${\bm{R}}^{(t)} = \beta (t){\bm{R}}_b$ and ${\bm{R}}_b = \mathds 1 {\bm{m}}' - {\bm{I}}$, the forward process can be written as where $(\bar{{\bm{Q}}}^{(t)})_{ij} = q(z^{(t)} = i \mid z^0 = j)$ and $\bar{\beta}^{(t)} = \int_0^t \beta (s)ds$.

Figures (8)

  • Figure 1: Overview of Cometh. During training, Cometh, unlike previous discrete-state diffusion models, transitions at any time $t \in [0,1]$, while during sampling, the step length is fixed to $\tau$. During sampling, we can additionally apply multiple corrector steps at $t-\tau$, which experimentally leads to better sample quality.
  • Figure 2: Comparison between our cosine noise schedule and the constant noise schedule proposed by campbell2022continuous. Both schedules are plotted using a rate constant $\alpha = 5$.
  • Figure 3: Training
  • Figure 4: Overview of DiGress graph transformer.
  • Figure 5: Samples from Cometh on Planar (top) and SBM (bottom)
  • ...and 3 more figures

Theorems & Definitions (20)

  • Proposition 1
  • Theorem 2: Informal
  • Proposition 3
  • Proposition 4
  • proof
  • Proposition 5
  • proof
  • Proposition 6
  • proof
  • Proposition 7
  • ...and 10 more