Diffusion Models for Graphs Benefit From Discrete State Spaces

TL;DR

This paper tackles discrete graph generation with diffusion models by replacing the traditional continuous Gaussian forward noise with discrete multinomial noise, ensuring every diffusion step yields a valid graph. It derives a tractable reverse process and trains via a variational objective, using a simple edgewise network to predict $p_{ heta}(m{A}_{0}|m{A}_{t})$ (with an alternative $L_{ ext{simple}}$ loss) and two architectures, EDP-GNN and PPGN, to model the reverse dynamics. Empirically, the discrete formulation yields higher quality samples (average MMD improved by about $1.5$) and dramatically faster sampling, reducing denoising steps from $1000$ to $32$ and delivering roughly a $30$-fold speedup across multiple graph datasets (Ego-small, Community-small, SBM-27, Planar-60). The results indicate that discrete diffusion is especially advantageous for larger graphs and that the proposed losses and architectures enable practical, scalable graph generation with strong distributional coverage. This work thus broadens diffusion-based graph generation by showing discrete-noise diffusion can outperform Gaussian counterparts in both quality and efficiency, with potential extensions to graphs with edge attributes.

Abstract

Denoising diffusion probabilistic models and score-matching models have proven to be very powerful for generative tasks. While these approaches have also been applied to the generation of discrete graphs, they have, so far, relied on continuous Gaussian perturbations. Instead, in this work, we suggest using discrete noise for the forward Markov process. This ensures that in every intermediate step the graph remains discrete. Compared to the previous approach, our experimental results on four datasets and multiple architectures show that using a discrete noising process results in higher quality generated samples indicated with an average MMDs reduced by a factor of 1.5. Furthermore, the number of denoising steps is reduced from 1000 to 32 steps, leading to a 30 times faster sampling procedure.

Figures (16)

• Figure 1: Sample Planar-60 and SBM-27 graphs generated by our approach (PPGN $L_{simple}$ - Middle) and the original Gaussian Score-matching approach (EDP-Score - Right).
• Figure 2: Average MMD compared to the number of denoising steps used on the Ego dataset for PPGN $L_{\text{simple}}$, which uses discrete noise and PPGN-Score, which uses Gaussian noise.
• Figure 3: Sample graphs from the training set of Ego-small dataset.
• Figure 4: Sample graphs generated with the model EDP-Score pmlr-v108-niu20a for the Ego-small dataset.
• Figure 5: Sample graphs generated with the PPGN $L_{\text{vb}}$ model for the Ego-small dataset.