DeFoG: Discrete Flow Matching for Graph Generation

TL;DR

DeFoG introduces Discrete Flow Matching for Graph Generation, a framework that decouples training and sampling in graph diffusion, enabling flexible, efficient sampling via a continuous-time Markov chain denoising process. By predicting marginal clean-node/edge distributions with a permutation-equivariant neural network and using a decoupled rate-matrix design at sampling time, it achieves state-of-the-art performance on synthetic and molecular graphs with only a fraction of the sampling steps required by diffusion models. The authors provide theoretical guarantees linking training objectives to sampling dynamics, and they thoroughly explore a rich design space—time distortions, target guidance, and stochastic priors—supported by extensive ablations and conditional generation experiments. Overall, DeFoG offers a scalable, flexible approach for high-quality graph generation with provable grounding and practical efficiency gains.

Abstract

Graph generative models are essential across diverse scientific domains by capturing complex distributions over relational data. Among them, graph diffusion models achieve superior performance but face inefficient sampling and limited flexibility due to the tight coupling between training and sampling stages. We introduce DeFoG, a novel graph generative framework that disentangles sampling from training, enabling a broader design space for more effective and efficient model optimization. DeFoG employs a discrete flow-matching formulation that respects the inherent symmetries of graphs. We theoretically ground this disentangled formulation by explicitly relating the training loss to the sampling algorithm and showing that DeFoG faithfully replicates the ground truth graph distribution. Building on these foundations, we thoroughly investigate DeFoG's design space and propose novel sampling methods that significantly enhance performance and reduce the required number of refinement steps. Extensive experiments demonstrate state-of-the-art performance across synthetic, molecular, and digital pathology datasets, covering both unconditional and conditional generation settings. It also outperforms most diffusion-based models with just 5-10% of their sampling steps.

Figures (20)

• Figure 1: (a) DeFoG enhances graph generation by introducing training-sampling decoupling, an orthogonal improvement within graph iterative refinement models, while preserving the sampling flexibility and inherent discreteness exploitation of prior SOTA models. (b) One node, $x^{(n)}$, is selected to illustrate both noising and denoising processes. For noising, DeFoG follows a straight path from the one-hot encoding $p_1$ of the clean node to the initial distribution $p_0$. For denoising, a network parameterized by $\theta$ predicts the marginal distributions of the clean graph, there the node's distribution ${\bm{p}^{\theta, (n)}_{1|t} (\cdot|G_t)}$ is used to compute its rate matrix $\bm{R}_t^{\theta,(n)}$ and, subsequently, its probability at the next time point $t+\Delta t$.
• Figure 2: DeFoG's improvements on efficiency.
• Figure 3: Effect of proposed sampling approaches on the Planar dataset. Higher values on the vertical axis correspond to more favorable values for both V.U.N. and Ratio.
• Figure 4: (a) The five distortion functions explored. (b) The resulting PDFs for the five distortion functions. For polydec, identity, and polyinc, they were computed in closed-form. For revcos and cos, they were simulated with $10^4$ repetitions.
• Figure 5: Examples of different rate matrices from the space of 3$\times$3 matrices that satisfy the detailed balance condition. Here $p_i$ denotes $p_{t|1}(i|z_1)$.

Theorems & Definitions (20)

• Corollary 1: Bounded estimation error of rate matrix for graphs
• Corollary 2: Bounded deviation of the generated graph distribution
• Lemma 3: Node Permutation Equivariance and Invariance Properties of DeFoG
• Lemma 10: Rate matrices for target guidance
• proof
• Theorem 11: Bounded estimation error of unconditional multivariate rate matrix
• proof
• Corollary 12: Bounded estimation error of unconditional rate matrix for graphs
• proof
• Lemma 5: Upper bound time derivative of conditional univariate rate matrix