GLAD: Improving Latent Graph Generative Modeling with Simple Quantization

TL;DR

GLAD addresses graph generation in latent space by designing a discrete, permutation-equivariant latent space and learning its prior with diffusion bridges adapted to constrained discrete domains. The model encodes graphs with an equivariant autoencoder, applies finite scalar quantization to obtain a discrete latent grid, and uses Z_G- and Pi-bridges plus a model bridge to generate new latent graphs, decoded via a set-based graph transformer. Empirical results on generic graphs and molecule datasets show GLAD achieves competitive or state-of-the-art performance, with strong reconstruction fidelity and notable gains in chemical and structural metrics such as NSPDK and FCD. The work demonstrates the value of discrete latent spaces and diffusion-based priors for holistic, permutation-invariant graph generation applicable to both structural graphs and molecular graphs, with ablations confirming the importance of quantization and prior choice.

Abstract

Learning graph generative models over latent spaces has received less attention compared to models that operate on the original data space and has so far demonstrated lacklustre performance. We present GLAD a latent space graph generative model. Unlike most previous latent space graph generative models, GLAD operates on a discrete latent space that preserves to a significant extent the discrete nature of the graph structures making no unnatural assumptions such as latent space continuity. We learn the prior of our discrete latent space by adapting diffusion bridges to its structure. By operating over an appropriately constructed latent space we avoid relying on decompositions that are often used in models that operate in the original data space. We present experiments on a series of graph benchmark datasets that demonstrates GLAD as the first equivariant latent graph generative method achieves competitive performance with the state of the art baselines.

Figures (6)

• Figure 1: (Top) We encode an annotated graph with node features (colored squares) to a set of node embeddings (colored diamonds). We visualize three studied graph-latent spaces: continuous-graph latent (C-G), continuous-node latent (C-N), and quantised latent (GLAD) spaces. The grey disks denote normal priors, the colored circles denote posterior distributions, and the green arrows denote pushing forces. GLAD distinctively maps the raw-node embeddings to discrete points on a uniformly quantized space (black dots). (Bottom) We decode a fully-connected pseudo latent graph (dashed-edge graph) formed by the set of quantized latent nodes $Z_{\mathcal{G}}$ (colored dots) to the original graph space. We leverage diffusion bridges to learn the graph-latent discrete distribution $\Pi$ constrained over the quantized-latent space $\mathbf{S}$. The colored paths represent an example of $Z_{\mathcal{G}}$-bridge.
• Figure 2: Graph transformer-based architectures. (Left) graph encoder, (Middle) graph decoder, (Right) model bridge drift. We denote $\oplus$ as pooling operator on a set of latent nodes, and $\cup$ as concatenation operator.
• Figure 3: Molecule reconstruction from different latent spaces on QM9. Atom (Left) and bond (Right) type test accuracies: i) continuous-graph latent representation (C-G). ii) continuous-node latent representation (C-N) iii) unquantized discrete-node latent space $\text{GLAD}^{\dagger}$, iv) quantised discrete-node latent space GLAD. Dashed back lines denote the percentage of dominant atom (Carbon) and bond (Single) types.
• Figure 4: Comparison of latent node distributions on the quantized-grid structure $\mathbf{S}$ of train and sampled molecules, QM9 (Left) and ZINC250k (Right).
• Figure 5: Visualization of generated samples on QM9 from GLAD.