Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quotient Normalized Maximum Likelihood Criterion for Learning Bayesian Network Structures

Tomi Silander, Janne Leppä-aho, Elias Jääsaari, Teemu Roos

TL;DR

This paper introduces the quotient normalized maximum likelihood ($qNML$) criterion for learning Bayesian network structures, addressing hyperparameter sensitivity and interpretability concerns of prior-based scores. $qNML$ is designed to be score equivalent, decomposable, and parameter-free, while leveraging Szpankowski–Weinberger approximations to remain computationally feasible. The authors prove key properties: score equivalence, consistency (via asymptotic alignment with BIC), and regularity, and show that $qNML$ coincides with the NML score for many network classes. Empirical results across synthetic benchmarks and real data demonstrate that $qNML$ yields parsimonious, predictive models and often ranks as a robust, safe choice compared with BIC, BDeu, and fNML. Overall, $qNML$ provides a theoretically grounded, practical alternative for BN structure learning with strong interpretability and competitive performance.

Abstract

We introduce an information theoretic criterion for Bayesian network structure learning which we call quotient normalized maximum likelihood (qNML). In contrast to the closely related factorized normalized maximum likelihood criterion, qNML satisfies the property of score equivalence. It is also decomposable and completely free of adjustable hyperparameters. For practical computations, we identify a remarkably accurate approximation proposed earlier by Szpankowski and Weinberger. Experiments on both simulated and real data demonstrate that the new criterion leads to parsimonious models with good predictive accuracy.

Quotient Normalized Maximum Likelihood Criterion for Learning Bayesian Network Structures

TL;DR

This paper introduces the quotient normalized maximum likelihood () criterion for learning Bayesian network structures, addressing hyperparameter sensitivity and interpretability concerns of prior-based scores. is designed to be score equivalent, decomposable, and parameter-free, while leveraging Szpankowski–Weinberger approximations to remain computationally feasible. The authors prove key properties: score equivalence, consistency (via asymptotic alignment with BIC), and regularity, and show that coincides with the NML score for many network classes. Empirical results across synthetic benchmarks and real data demonstrate that yields parsimonious, predictive models and often ranks as a robust, safe choice compared with BIC, BDeu, and fNML. Overall, provides a theoretically grounded, practical alternative for BN structure learning with strong interpretability and competitive performance.

Abstract

We introduce an information theoretic criterion for Bayesian network structure learning which we call quotient normalized maximum likelihood (qNML). In contrast to the closely related factorized normalized maximum likelihood criterion, qNML satisfies the property of score equivalence. It is also decomposable and completely free of adjustable hyperparameters. For practical computations, we identify a remarkably accurate approximation proposed earlier by Szpankowski and Weinberger. Experiments on both simulated and real data demonstrate that the new criterion leads to parsimonious models with good predictive accuracy.
Paper Structure (24 sections, 20 theorems, 51 equations, 3 figures, 3 tables)

This paper contains 24 sections, 20 theorems, 51 equations, 3 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. BAYESIAN NETWORKS
  3. Likelihood
  4. Bayesian Structure Learning
  5. Problems, Solutions and Problems
  6. QUOTIENT NML SCORE
  7. qNML Is Score Equivalent
  8. qNML is Consistent
  9. qNML Equals NML for Many Models
  10. qNML is Regular
  11. EXPERIMENTAL RESULTS
  12. Finding Generating Structure
  13. Prediction and Parsimony
  14. CONCLUSION
  15. Score equivalency proof
  16. ...and 9 more sections

Key Result

Theorem 1

Let $G$ and $G'$ be two Bayesian network structures that differ only by a single covered arc reversal, i.e., the arc from $A$ to $B$ in $G$ has been reversed in $G'$ to point from $B$ to $A$, then

Figures (3)

  • Figure 1: Number of parameters in a breast cancer model as a function of sample size for different model selection criteria.
  • Figure 2: Sample size versus SHD with data generated from real world DAGs.
  • Figure 3: Average ranks for the scoring functions in structure learning experiments.

Theorems & Definitions (38)

  • Theorem 1
  • proof
  • Theorem 2
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Theorem 3
  • proof
  • Definition 1
  • ...and 28 more