Pruning-Based Extraction of Descriptions from Probabilistic Circuits

TL;DR

The paper tackles the challenge of inspecting learned probabilistic circuits by extracting a discriminative CNF model that highlights high-density regions of the circuit. It introduces PUTPUT, a two-step pruning method (pruning sum nodes followed by input nodes) and a novel description length, aggregated entropy, to produce compact, readable formulas. Empirical results on a real-world music playlist task and open datasets show PUTPUT achieves higher F1 scores and substantially lower aggregated entropy than competitive PU-based methods. The approach enhances safety and interpretability of concept learning with probabilistic circuits and broadly applies to pattern mining and explainable AI tasks beyond music playlist generation.

Abstract

Concept learning is a general task with applications in various domains. As a motivating example we consider the application of music playlist generation, where a playlist is represented as a concept (e.g., `relaxing music') rather than as a fixed collection of songs. In this work we use a probabilistic circuit to learn a concept from positively labelled and unlabelled examples. While these circuits form an attractive tractable model for this task, it is challenging for a domain expert to inspect and analyse them, which impedes their use within certain applications. We propose to resolve this by converting a learned probabilistic circuit into a logic-based discriminative model that covers the high density regions of the circuit. That is, those regions the circuit classifies as certainly being part of the learned concept. As part of this approach we present two contributions: PUTPUT, an algorithm to prune low density regions from a probabilistic circuit while considering both the F1-score and a newly proposed description length that we call aggregated entropy. Our experiments demonstrate the effectiveness of our approach in providing discriminative models, outperforming competitors on the music playlist generation task and similar datasets.

Figures (7)

• Figure 1: Overview of our approach. A (large) probabilistic circuit over variables Style(S), Mood(M) and Version(V) is pruned to only contain the high density regions. This smaller circuit is transformed into a logic formula, acting as a discriminative model that is easier to inspect and verify by a domain expert.
• Figure 2: Generation of a probability with given input $\{m_1,v_2\}$. We marginalise over $S$, as no value is assigned. The dashed lines show the parts of the circuit where a zero probability is propagated.
• Figure 3: Logical formula derived from the probabilistic circuit in Figure \ref{['fig:intro-box']}a.
• Figure 4: The result after pruning the probabilistic circuit from Figure \ref{['fig:intro-box']}a, with the generative significance method set to eliminate the five nodes with the lowest top down probabilities. These values are shown numerically inside the nodes.
• Figure 5: The resulting circuit after applying step 2 of PUTPUT, with $\mathcal{E}_{HDR}=\{(s_1,m_1,v_1), (s_1,m_1,v_2),(s_1,m_2,v_1)\}$

Theorems & Definitions (4)

• Definition 1: Entropy of a variable within a clause
• Definition 2: Aggregated entropy of a clause
• Definition 3: Aggregated entropy of a CNF
• Example 1