Probabilistic Graph Circuits: Deep Generative Models for Tractable Probabilistic Inference over Graphs

TL;DR

The paper introduces Probabilistic Graph Circuits (PGCs), a tractable deep generative framework for graphs that enables exact probabilistic inference over variable-size graphs. By extending probabilistic circuits to graphs and enforcing or approximating permutation invariance via marginalization or sorting, PGCs perform exact or near-exact marginalization, conditioning, and expectations without query-specific architectures. The work proves tractability conditions for $\mathbb{S}_n$-invariance, proposes marginalization padding to handle variable graph sizes, and analyzes inherent versus sorting-based invariance, demonstrating competitive molecular graph generation against intractable DGMs and enabling tractable conditional graph inference. Empirically, PGCs achieve strong performance on QM9 and Zinc250k across unconditional generation and conditional tasks, while highlighting limitations in valency validity and proposing avenues like rejection sampling and richer cross-component connections for future improvements.

Abstract

Deep generative models (DGMs) have recently demonstrated remarkable success in capturing complex probability distributions over graphs. Although their excellent performance is attributed to powerful and scalable deep neural networks, it is, at the same time, exactly the presence of these highly non-linear transformations that makes DGMs intractable. Indeed, despite representing probability distributions, intractable DGMs deny probabilistic foundations by their inability to answer even the most basic inference queries without approximations or design choices specific to a very narrow range of queries. To address this limitation, we propose probabilistic graph circuits (PGCs), a framework of tractable DGMs that provide exact and efficient probabilistic inference over (arbitrary parts of) graphs. Nonetheless, achieving both exactness and efficiency is challenging in the permutation-invariant setting of graphs. We design PGCs that are inherently invariant and satisfy these two requirements, yet at the cost of low expressive power. Therefore, we investigate two alternative strategies to achieve the invariance: the first sacrifices the efficiency, and the second sacrifices the exactness. We demonstrate that ignoring the permutation invariance can have severe consequences in anomaly detection, and that the latter approach is competitive with, and sometimes better than, existing intractable DGMs in the context of molecular graph generation.

Figures (20)

• Figure 1: An example of a PGC for undirected acyclic graphs. (a) We consider a graph $\mathbf{G}$ represented by a feature matrix, $\mathbf{X}$, and an adjacency tensor, $\mathbf{A}$, such that each instance of $\mathbf{G}$ (highlighted in green and blue) has a random number of nodes, $n\in(0,1,\ldots,m)$, where $m$ is a fixed maximum number of nodes. The empty places (white) are not included in the training data. (b) The main building block of PGCs is the $n$-conditioned joint distribution over $\mathbf{X}^n$ and $\mathbf{L}^n$, the latter of which is a flattened lower triangular part of $\mathbf{A}^n$. $\mathbf{X}^n$ and $\mathbf{L}^n$ are used as input into the node-PC and edge-PC, respectively. The empty places are marginalized out (grey). The outputs of these two PCs are passed through the product layer with $n_c$ units and the sum layer with a single unit.
• Figure 2: $\mathbb{S}_n$-sensitivity of PGCs. The histograms of $\log p(\mathbf{G})$ and $\log p(\mathbf{G}|\bm{\pi}_c)$ (left) and the AUC (right) for different $\mathbb{S}_n$-invariance mechanisms of PGCs in the context of anomaly detection.
• Figure 3: Conditional generation on the Zinc250k dataset. The yellow area highlights the known part of the molecule. There is one such known part per row. Each column corresponds to a new molecule generated conditionally on the known part.
• Figure 4: An example of the marginal evidence query over a 4-node graph. (a) An instantiation of the omni-compatible GC \ref{['eq:h']} over a graph $\mathbf{G}^4$ with targetting the 2nd node, $\mathbf{s}\coloneqq\{2\}$. The orange color highlights the targeted node and its associated edges. (b) A visual representation of $\mathbf{G}$, where the targeted node and its associated edges, which correspond to (a), are highlighted in orange. (c) After performing the marginal evidence query over the 2nd node of $\mathbf{G}^4$, we obtain a new 3-node evidence graph $\mathbf{g}$.
• Figure 5: An inherently $\mathbb{S}_n$-invariant PGC through the conditional i.i.d. assumption. The $n$-conditioned part of a PGC that is tractable and inherently $\mathbb{S}_n$-invariant, as formulated in \ref{['prop:factorized-pgcs']}. The input units with the same color share the parameterization and correspond to the product terms in \ref{['eq:iid-components']}.

Theorems & Definitions (17)

• Definition 1
• Definition 2
• Definition 3: Graph Scope
• Definition 4: Graph Circuit
• Definition 5: Probabilistic Graph Circuit
• Definition 6
• Proposition 1
• Definition 7
• Proposition 2
• Proposition 3