Restructuring Tractable Probabilistic Circuits

TL;DR

This work tackles restructuring structured-decomposable probabilistic circuits (PCs) to follow a target vtree, enabling operations across heterogeneous PC structures. It develops a latent-variable interpretation that casts PCs as tree-shaped Bayesian networks, then recovers a recursive construction that yields a PC conforming to any target vtree while preserving the represented distribution. Key contributions include a general restructuring algorithm, tractable (poly/quasi-poly) results for multiplying contiguous PCs with different vtrees, and a depth-reduction method that produces log-depth PCs with tight size bounds, all while maintaining structured decomposability. These ideas open the door to training with more flexible PC structures and performing inference more efficiently by restructuring at inference time, with practical implications for combining PCs with other structured constraints (e.g., PCFGs, SDDs) and improving parallelism through depth reduction.

Abstract

Probabilistic circuits (PCs) are a unifying representation for probabilistic models that support tractable inference. Numerous applications of PCs like controllable text generation depend on the ability to efficiently multiply two circuits. Existing multiplication algorithms require that the circuits respect the same structure, i.e. variable scopes decomposes according to the same vtree. In this work, we propose and study the task of restructuring structured(-decomposable) PCs, that is, transforming a structured PC such that it conforms to a target vtree. We propose a generic approach for this problem and show that it leads to novel polynomial-time algorithms for multiplying circuits respecting different vtrees, as well as a practical depth-reduction algorithm that preserves structured decomposibility. Our work opens up new avenues for tractable PC inference, suggesting the possibility of training with less restrictive PC structures while enabling efficient inference by changing their structures at inference time.

Figures (6)

• Figure 1: Fig. \ref{['fig:vtree']} shows a vtree $V$ for some contiguous PC $\mathscr{A}$; Fig. \ref{['fig:bn']} shows a Bayesian network representation $G_{\mathscr{A}}$ for $\mathscr{A}$; Fig. \ref{['fig:labelling']} shows a valid labelling of vtree $W$ with respect to $G_{\mathscr{A}}$.
• Figure 2: Recursive construction of vectors of sum nodes representing $p(\bm{X}_w \:\vert\:\bm{C}_w)$
• Figure 3: Summary of restructuring results; the top/bottom rows indicate the source/target structures. Blue and bold arrows indicate novel complexity results. For the two other arrows that were known to be polynomial-time, our approach yields more efficient algorithms amenable to practical implementation.
• Figure 4: $G_{\mathscr{A}}$ for $\mathscr{A}$ with a linear vtree
• Figure 5: Example of restructuring labeling.

Theorems & Definitions (46)

• Definition 2.1: Probabilistic Circuit
• Definition 2.2: Smoothness and Decomposability
• Definition 2.3: Vtree
• Definition 2.4: Structured Decomposability
• Definition 3.1: Augmented PC
• Proposition 3.1
• Theorem 3.2
• Definition 3.3: Blocked Path
• Definition 3.4: Cover
• Proposition 3.5: geiger1990d