Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GraphSPNs: Sum-Product Networks Benefit From Canonical Orderings

Milan Papež, Martin Rektoris, Václav Šmídl, Tomáš Pevný

TL;DR

GraphSPNs extend sum-product networks to graphs by padding to a fixed size and enforcing permutation invariance, enabling exact and efficient probabilistic inference over graphs G=(X,A) with varying node counts. They explore exact, canonical (sorting), and approximate invariance via $k$-ary subgraphs and random sampling, plus virtual node-padding to manage variable graph sizes. Empirical results on QM9 show GraphSPNs can generate chemically valid and novel molecules, with the canonical ordering variant delivering the strongest practical performance by reducing multimodality. The work provides a scalable framework for exact probabilistic queries on graphs and offers a principled approach to molecular graph generation and conditional inference.

Abstract

Deep generative models have recently made a remarkable progress in capturing complex probability distributions over graphs. However, they are intractable and thus unable to answer even the most basic probabilistic inference queries without resorting to approximations. Therefore, we propose graph sum-product networks (GraphSPNs), a tractable deep generative model which provides exact and efficient inference over (arbitrary parts of) graphs. We investigate different principles to make SPNs permutation invariant. We demonstrate that GraphSPNs are able to (conditionally) generate novel and chemically valid molecular graphs, being competitive to, and sometimes even better than, existing intractable models. We find out that (Graph)SPNs benefit from ensuring the permutation invariance via canonical ordering.

GraphSPNs: Sum-Product Networks Benefit From Canonical Orderings

TL;DR

GraphSPNs extend sum-product networks to graphs by padding to a fixed size and enforcing permutation invariance, enabling exact and efficient probabilistic inference over graphs G=(X,A) with varying node counts. They explore exact, canonical (sorting), and approximate invariance via -ary subgraphs and random sampling, plus virtual node-padding to manage variable graph sizes. Empirical results on QM9 show GraphSPNs can generate chemically valid and novel molecules, with the canonical ordering variant delivering the strongest practical performance by reducing multimodality. The work provides a scalable framework for exact probabilistic queries on graphs and offers a principled approach to molecular graph generation and conditional inference.

Abstract

Deep generative models have recently made a remarkable progress in capturing complex probability distributions over graphs. However, they are intractable and thus unable to answer even the most basic probabilistic inference queries without resorting to approximations. Therefore, we propose graph sum-product networks (GraphSPNs), a tractable deep generative model which provides exact and efficient inference over (arbitrary parts of) graphs. We investigate different principles to make SPNs permutation invariant. We demonstrate that GraphSPNs are able to (conditionally) generate novel and chemically valid molecular graphs, being competitive to, and sometimes even better than, existing intractable models. We find out that (Graph)SPNs benefit from ensuring the permutation invariance via canonical ordering.
Paper Structure (31 sections, 1 theorem, 7 equations, 5 figures, 1 table)

This paper contains 31 sections, 1 theorem, 7 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Sum-product networks.
  4. Permutation invariance.
  5. Problem definition.
  6. Graph Sum-Product Networks
  7. Virtual node-padding.
  8. Exact permutation invariance.
  9. Sorting.
  10. Approximate permutation invariance.
  11. $k$-ary sub-graphs.
  12. Random sampling.
  13. Experiments
  14. Molecule generation.
  15. Dataset.
  16. ...and 16 more sections

Key Result

Proposition 1

(Tractability of GraphSPNs.) Let $p(G)$ be a GraphSPN eq:graphspn satisfying Assumptions ass:p-constraints, and let $f(G)$ be a function satisfying Assumption ass:f-constraints. Then, the integral eq:expectation is tractable.

Figures (5)

  • Figure 1: Graph representation. (a) Let $G$ be a graph represented by a feature matrix, $X\in\mathcal{X}^n$, and an adjacency tensor, $A\in\mathcal{A}^{n\times n}$. We consider each instance of $G$ (highlighted in green and blue) to have a random number of nodes, $n\in(0,1,\ldots,m)$, but we expect it to have at most $m$ nodes. The remaining places (white) are empty and are not included in the training data. (b) Virtual node-padding fills in the empty places with virtual nodes (grey), which requires us to extend $\mathcal{X}$ by an extra category, $\mathcal{X}\coloneqq(0,1)^{q+1}$.
  • Figure 2: Conditional molecule generation on the QM9 dataset. The blue area highlights the known part of the molecule. There is one such known part per row. Each column corresponds to a new molecule that is generated conditionally on the known part.
  • Figure 3: An illustration of the key principles behind various GraphSPNs. (a) The exact permutation invariance first computes $p^{\text{spn}}_{m,n}$ for all $n!$ permutations and then averages the results in \ref{['eq:exact']}. (b) The $k$-ary permutation invariance approximates the exact invariance (a) by computing $p^{\text{spn}}_{k,n}$ (with the root scope of size $k(k+1)$) for all ways to choose $k$-node sub-graphs from the $n$-node original graph, $G$, without repetition and with the order, and then averaging the $M=n!/(n-k)!<n!$ results in \ref{['eq:kary']}. (c) The random sampling approximates the exact invariance (a) by computing the average only for $N<n!$ permutations of $G$ in \ref{['eq:random']}. (d) The sorting approach is also exact, but it simplifies the target data distribution by first imposing the same canonical ordering of $G$ and then computing $p^{\text{spn}}_{m,n}$, as displayed in \ref{['eq:sorting']}.
  • Figure 4: An example of a tractable inference query over a graph. (a) An instantiation of the omni-compatible function \ref{['eq:f']} over a graph $G$ for $\mathbf{a}\coloneqq\{2\}$. The blue color highlights the targeted node and its associated edges. (b) A visual and matrix representation of $G$, where the targeted node and its associated edges, which correspond to (a), are highlighted in blue. (c) After targeting the node and its connected edges, we can perform, e.g., the marginal query and obtain a marginal graph $\overline{G}$.
  • Figure 5: Unconditional molecule generation on the QM9 dataset.

Theorems & Definitions (1)

  • Proposition 1