Neural Probabilistic Circuits: Enabling Compositional and Interpretable Predictions through Logical Reasoning

TL;DR

This work tackles the interpretability gap in end-to-end deep networks by introducing Neural Probabilistic Circuits (NPCs), which couple a neural attribute recognizer with a symbolic, tractable probabilistic circuit to perform predictions via logical reasoning over attributes. They propose a three-stage training pipeline—attribute recognition, circuit construction (data-driven or knowledge-injected), and joint optimization—and prove a compositional error bound tying overall performance to module errors. The framework also offers interpretable explanations through Most Probable Explanations and Counterfactual Explanations, and demonstrates competitive accuracy on four benchmarks while enhancing transparency. The results suggest NPCs can achieve strong task performance with explicit, human-understandable reasoning, and outline limitations and future directions to further close interpretability-performance gaps in complex domains.

Abstract

End-to-end deep neural networks have achieved remarkable success across various domains but are often criticized for their lack of interpretability. While post hoc explanation methods attempt to address this issue, they often fail to accurately represent these black-box models, resulting in misleading or incomplete explanations. To overcome these challenges, we propose an inherently transparent model architecture called Neural Probabilistic Circuits (NPCs), which enable compositional and interpretable predictions through logical reasoning. In particular, an NPC consists of two modules: an attribute recognition model, which predicts probabilities for various attributes, and a task predictor built on a probabilistic circuit, which enables logical reasoning over recognized attributes to make class predictions. To train NPCs, we introduce a three-stage training algorithm comprising attribute recognition, circuit construction, and joint optimization. Moreover, we theoretically demonstrate that an NPC's error is upper-bounded by a linear combination of the errors from its modules. To further demonstrate the interpretability of NPC, we provide both the most probable explanations and the counterfactual explanations. Empirical results on four benchmark datasets show that NPCs strike a balance between interpretability and performance, achieving results competitive even with those of end-to-end black-box models while providing enhanced interpretability.

Figures (9)

• Figure 1: Model architecture of an NPC, consisting of an attribute recognition model and a task predictor. The attribute recognition model is a neural network $f(X;\theta)$ which takes an image $X$ as input and outputs $K$ probability vectors $\{\Pr_{\theta}(A_k\mid X)\}_{k=1}^K$. The task predictor is a probabilistic circuit $f_S(Y, A_{1:K}; w)$ taking an instance of attributes as input and providing the conditional probability $\Pr_w(Y\mid A_{1:K})$. By leveraging these relations between classes and attributes alongside the probability distributions of various attributes, NPC produces the probability vector $\Pr_{\theta, w}(Y\mid X)$.
• Figure 2: Illustration of a probabilistic circuit constructed by the knowledge-injected approach. The circuit encodes the set of weighted logical rules: $\{w_1 \cdot \left(\mathbb{I}\left(A_1=0\right) \wedge \mathbb{I}\left(A_2=0\right) \wedge \mathbb{I}\left(Y=0\right) \right),~w_2 \cdot \left(\mathbb{I}(A_1=1) \wedge \mathbb{I}(A_2=1) \wedge \mathbb{I}(Y=1) \right),~w_3 \cdot \left( \mathbb{I}\left(A_1=0\right) \wedge \mathbb{I}\left(A_2=1\right) \wedge \mathbb{I}\left(Y=1\right)\right)\}$.
• Figure 3: NPC classification accuracy on the GTSRB and MNIST-Addition datasets with attribute exclusions. The gray bars indicate inference with all attributes, while non-gray bars indicate inference with one particular attribute excluded.
• Figure 4: Classification accuracy of NPC (Data) and NPC (Knowledge) on the four benchmark datasets. The blue bars indicate performance prior to joint optimization, while the orange bars illustrate performance after applying joint optimization.
• Figure 5: Examples from the four benchmark datasets. Each example includes an image, the ground-truth class label, the ground-truth attribute labels, the NPC (Data) class prediction, and, lastly, the corresponding MPE. These examples illustrate MPEs that align with the ground-truth attribute labels. The CelebA image is redacted in compliance with its terms of use.

Theorems & Definitions (8)

• Proposition 1
• proof
• Theorem 2: Compositional Error
• proof : Proof Sketch
• Definition 1: Most Probable Explanations
• Definition 2: Alignment Rate
• Definition 3: Counterfactual Explanations
• Definition 4: Correction Rate