Sum-Product-Set Networks: Deep Tractable Models for Tree-Structured Graphs
Milan Papež, Martin Rektoris, Tomáš Pevný, Václav Šmídl
TL;DR
This work addresses probabilistic modeling over tree-structured graphs such as JSON/XML by introducing Sum-Product-Set Networks (SPSNs), which extend probabilistic circuits with Random Finite Sets to handle variable-sized trees and heterogeneity. SPSNs define a density $p(T)$ over tree-structured data via a parameterized computational graph $\mathcal{G}$ and a scope function $\psi$, incorporating a set unit to model finite random sets and enabling exact marginal inference. The authors establish tractability under structural constraints, analyze exchangeability properties, and demonstrate competitive performance against intractable neural-network–based models, while offering robustness to missing data through tractable marginalization. This methodology provides a principled, scalable, and exact-inference alternative for learning and querying distributions over structured graphs, with practical impact on domains requiring reliable probabilistic reasoning over JSON/XML-like data. Key steps include constructing SPSN blocks from data schemas, enforcing smoothness and decomposability to guarantee closed-form inference, and leveraging finite random-set cardinalities to capture variable subtree counts, all while proving partial exchangeability and providing empirical evidence on real-world datasets.
Abstract
Daily internet communication relies heavily on tree-structured graphs, embodied by popular data formats such as XML and JSON. However, many recent generative (probabilistic) models utilize neural networks to learn a probability distribution over undirected cyclic graphs. This assumption of a generic graph structure brings various computational challenges, and, more importantly, the presence of non-linearities in neural networks does not permit tractable probabilistic inference. We address these problems by proposing sum-product-set networks, an extension of probabilistic circuits from unstructured tensor data to tree-structured graph data. To this end, we use random finite sets to reflect a variable number of nodes and edges in the graph and to allow for exact and efficient inference. We demonstrate that our tractable model performs comparably to various intractable models based on neural networks.