Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Greedy equivalence search for nonparametric graphical models

Bryon Aragam

TL;DR

The paper addresses structure learning for directed acyclic graphs in fully nonparametric settings by replacing the traditional BIC score with Bayesian model selection tests that compare local CPD neighbourhoods. It develops a modified GES that operates via decomposable, globally consistent tests based on posterior odds, enabling a consistent search over Markov equivalence classes under mild Lipschitz smoothness assumptions. The authors prove that, under these conditions, the modified GES recovers a perfect map of the true distribution (or a minimal model in the absence of faithfulness), thereby providing a general nonparametric consistency theorem that unifies and extends parametric results. The framework leverages CPDAG neighborhoods, Dirichlet process mixtures for CPDs, and Le Cam-type complexity control to deliver a practical, theoretically grounded approach to nonparametric structure learning with broad applicability to spline densities, smooth nonparanormal models, and related nonparametric DAG families.

Abstract

One of the hallmark achievements of the theory of graphical models and Bayesian model selection is the celebrated greedy equivalence search (GES) algorithm due to Chickering and Meek. GES is known to consistently estimate the structure of directed acyclic graph (DAG) models in various special cases including Gaussian and discrete models, which are in particular curved exponential families. A general theory that covers general nonparametric DAG models, however, is missing. Here, we establish the consistency of greedy equivalence search for general families of DAG models that satisfy smoothness conditions on the Markov factorization, and hence may not be curved exponential families, or even parametric. The proof leverages recent advances in nonparametric Bayes to construct a test for comparing misspecified DAG models that avoids arguments based on the Laplace approximation. Nonetheless, when the Laplace approximation is valid and a consistent scoring function exists, we recover the classical result. As a result, we obtain a general consistency theorem for GES applied to general DAG models.

Greedy equivalence search for nonparametric graphical models

TL;DR

The paper addresses structure learning for directed acyclic graphs in fully nonparametric settings by replacing the traditional BIC score with Bayesian model selection tests that compare local CPD neighbourhoods. It develops a modified GES that operates via decomposable, globally consistent tests based on posterior odds, enabling a consistent search over Markov equivalence classes under mild Lipschitz smoothness assumptions. The authors prove that, under these conditions, the modified GES recovers a perfect map of the true distribution (or a minimal model in the absence of faithfulness), thereby providing a general nonparametric consistency theorem that unifies and extends parametric results. The framework leverages CPDAG neighborhoods, Dirichlet process mixtures for CPDs, and Le Cam-type complexity control to deliver a practical, theoretically grounded approach to nonparametric structure learning with broad applicability to spline densities, smooth nonparanormal models, and related nonparametric DAG families.

Abstract

One of the hallmark achievements of the theory of graphical models and Bayesian model selection is the celebrated greedy equivalence search (GES) algorithm due to Chickering and Meek. GES is known to consistently estimate the structure of directed acyclic graph (DAG) models in various special cases including Gaussian and discrete models, which are in particular curved exponential families. A general theory that covers general nonparametric DAG models, however, is missing. Here, we establish the consistency of greedy equivalence search for general families of DAG models that satisfy smoothness conditions on the Markov factorization, and hence may not be curved exponential families, or even parametric. The proof leverages recent advances in nonparametric Bayes to construct a test for comparing misspecified DAG models that avoids arguments based on the Laplace approximation. Nonetheless, when the Laplace approximation is valid and a consistent scoring function exists, we recover the classical result. As a result, we obtain a general consistency theorem for GES applied to general DAG models.

Paper Structure

This paper contains 19 sections, 7 theorems, 28 equations, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Overview
  4. Preliminaries
  5. Notation
  6. Graphical models
  7. Function spaces and smoothness
  8. Priors and posteriors
  9. Greedy equivalence search
  10. Original GES
  11. Tests
  12. Modified GES
  13. Bayesian model selection for nonparametric graphical models
  14. Bayesian tests
  15. Prior assumptions
  16. ...and 4 more sections

Key Result

Lemma 3.1

If a test $\varphi$ is 1-globally consistent and decomposable, then it is locally consistent.

Theorems & Definitions (17)

  • Definition 2.1
  • Remark 2.1
  • Remark 2.2
  • Definition 2.2
  • Definition 3.1
  • Definition 3.2
  • Definition 3.3
  • Definition 3.4
  • Lemma 3.1
  • Remark 3.1
  • ...and 7 more