Principled Latent Diffusion for Graphs via Laplacian Autoencoders

TL;DR

LG-Flow is proposed, a latent graph diffusion framework that directly overcomes obstacles of quadratic complexity in the number of nodes, and achieves competitive results against state-of-the-art graph diffusion models, while achieving up to $1000\times-up speed-up.

Abstract

Graph diffusion models achieve state-of-the-art performance in graph generation but suffer from quadratic complexity in the number of nodes -- and much of their capacity is wasted modeling the absence of edges in sparse graphs. Inspired by latent diffusion in other modalities, a natural idea is to compress graphs into a low-dimensional latent space and perform diffusion there. However, unlike images or text, graph generation requires nearly lossless reconstruction, as even a single error in decoding an adjacency matrix can render the entire sample invalid. This challenge has remained largely unaddressed. We propose LG-Flow, a latent graph diffusion framework that directly overcomes these obstacles. A permutation-equivariant autoencoder maps each node into a fixed-dimensional embedding from which the full adjacency is provably recoverable, enabling near-lossless reconstruction for both undirected graphs and DAGs. The dimensionality of this latent representation scales linearly with the number of nodes, eliminating the quadratic bottleneck and making it feasible to train larger and more expressive models. In this latent space, we train a Diffusion Transformer with flow matching, enabling efficient and expressive graph generation. Our approach achieves competitive results against state-of-the-art graph diffusion models, while achieving up to $1000\times$ speed-up.

Figures (4)

• Figure 1: Overview of the LG-VAE. The encoder $\mathcal{E}$ (left) encodes structure via $\phi$ and node/edge labels, then aggregates them with $\rho$. Latents $\mathbold{Z}$ are sampled using the reparameterization trick and passed to the decoder $\mathcal{D}$ (right). Node labels are decoded directly, while adjacency is decoded by (a) computing scores via bilinear dot products for $b+1$ heads, (b) processing scores with a row-wise DeepSet, and (c) concatenating outputs across heads. The final adjacency $\hat{\mathbold{A}}$ is obtained with an argmax.
• Figure 2: Overview of LG-Flow. During training (top), the frozen encoder maps graphs into latent representations $\mathbold{Z}$. Noisy latents $\mathbold{Z}_t$ are sampled along a linear interpolation path and passed through the DiT, which predicts ${v_{\mathbold{\theta}}}$ and is optimized with the conditional flow matching loss. During sampling (bottom), noise is generated and iteratively denoised using the trained DiT. The final latents are then decoded into synthetic graphs using the frozen decoder.
• Figure 3: We compare molecular validity and sampling time. The ideal lies in the top-left corner, with high validity and low cost. Our method offers a better trade-off than prior graph-space diffusion models.
• Figure 4: Example of failure case for the decoder. Solid lines denote original edges, while the dashed line denotes a structurally correct prediction that will be considered as incorrect by the loss function.

Theorems & Definitions (7)

• Theorem 1
• Lemma 2
• proof
• Lemma 3
• proof
• Theorem 4
• proof