Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Unified Framework for Human-Allied Learning of Probabilistic Circuits

Athresh Karanam, Saurabh Mathur, Sahil Sidheekh, Sriraam Natarajan

TL;DR

This work proposes a unified framework to infuse domain knowledge into learning probabilistic circuits (PCs) by encoding various knowledge forms as differentiable domain constraints. Learning becomes a constrained optimization problem where the PC log-likelihood $\mathcal{L}(\mathcal{M}, \mathcal{D})$ is maximized while ensuring constraint satisfaction via a penalty function $\zeta$, with iterative penalty weights $\lambda_t$ updated by a factor $\gamma$ (Algorithm 1). Knowledge is represented as equal or inequality constraints on marginal and conditional queries, encompassing generalization, context-specific independence, monotonic influence, class-imbalance, and privileged information, enabling six instantiations. The framework is validated on synthetic, benchmark, and real-world data using RatSPN and EinsumNet, showing faithful constraint integration and improved generalization over purely data-driven learning, with demonstrated robustness to constraint noise and hyperparameters. Overall, the approach offers a simple, effective pathway for domain-expert guided PC learning in data-scarce, knowledge-rich settings, with broad applicability and potential extensions to relational data and PC structure learning.

Abstract

Probabilistic Circuits (PCs) have emerged as an efficient framework for representing and learning complex probability distributions. Nevertheless, the existing body of research on PCs predominantly concentrates on data-driven parameter learning, often neglecting the potential of knowledge-intensive learning, a particular issue in data-scarce/knowledge-rich domains such as healthcare. To bridge this gap, we propose a novel unified framework that can systematically integrate diverse domain knowledge into the parameter learning process of PCs. Experiments on several benchmarks as well as real world datasets show that our proposed framework can both effectively and efficiently leverage domain knowledge to achieve superior performance compared to purely data-driven learning approaches.

A Unified Framework for Human-Allied Learning of Probabilistic Circuits

TL;DR

This work proposes a unified framework to infuse domain knowledge into learning probabilistic circuits (PCs) by encoding various knowledge forms as differentiable domain constraints. Learning becomes a constrained optimization problem where the PC log-likelihood is maximized while ensuring constraint satisfaction via a penalty function , with iterative penalty weights updated by a factor (Algorithm 1). Knowledge is represented as equal or inequality constraints on marginal and conditional queries, encompassing generalization, context-specific independence, monotonic influence, class-imbalance, and privileged information, enabling six instantiations. The framework is validated on synthetic, benchmark, and real-world data using RatSPN and EinsumNet, showing faithful constraint integration and improved generalization over purely data-driven learning, with demonstrated robustness to constraint noise and hyperparameters. Overall, the approach offers a simple, effective pathway for domain-expert guided PC learning in data-scarce, knowledge-rich settings, with broad applicability and potential extensions to relational data and PC structure learning.

Abstract

Probabilistic Circuits (PCs) have emerged as an efficient framework for representing and learning complex probability distributions. Nevertheless, the existing body of research on PCs predominantly concentrates on data-driven parameter learning, often neglecting the potential of knowledge-intensive learning, a particular issue in data-scarce/knowledge-rich domains such as healthcare. To bridge this gap, we propose a novel unified framework that can systematically integrate diverse domain knowledge into the parameter learning process of PCs. Experiments on several benchmarks as well as real world datasets show that our proposed framework can both effectively and efficiently leverage domain knowledge to achieve superior performance compared to purely data-driven learning approaches.
Paper Structure (11 sections, 1 theorem, 7 equations, 1 figure)

This paper contains 11 sections, 1 theorem, 7 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Background
  3. Probabilistic Circuits (PCs)
  4. Knowledge-based Learning.
  5. Learning PCs with Domain Knowledge
  6. Encoding knowledge as constraints
  7. Defining the penalty function
  8. Experimental Evaluation
  9. Experimental Setup.
  10. Results
  11. Discussion

Key Result

Theorem 1

If the PC $\mathcal{M}$ is smooth, decomposable, and deterministic, and $\zeta$ is a differentiable, concave function of $\theta,$ then algorithm 1 is guaranteed to converge to the optimal feasible solution of equation eq:opt3, if one exists, as $t_\text{max} \rightarrow{} \infty.$

Figures (1)

  • Figure 1: 3D Helix dataset: Visualization of the 3D Helix dataset. The train dataset (a) consists of a single helix of length $2\pi$ with Gaussian noise added to it. The test dataset (b) consists of a helix of length $4\pi$ with Gaussian noise added to it. (c) and (d) visualizes 1000 samples generated from EinsumNet with and without incorporating the Generalization Constraint (GC), respectively.

Theorems & Definitions (5)

  • Definition 1
  • Definition 2
  • Definition 3
  • Theorem 1
  • proof