Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal estimation of Gaussian (poly)trees

Yuhao Wang, Ming Gao, Wai Ming Tai, Bryon Aragam, Arnab Bhattacharyya

TL;DR

This work provides a unified finite-sample analysis for learning Gaussian trees and polytrees, addressing both distribution learning (via KL distance) and structure learning (exact recovery). It introduces a Chow-Liu–type method for distribution learning and a PC-tree algorithm for polytree structure learning based on partial correlations, with explicit upper and matching lower bounds across non-realizable, realizable, and faithful scenarios. The results reveal phase transitions in sample complexity between distribution and structure learning and establish minimax optimality under strong-tree-faithfulness. Empirically, PC-Tree demonstrates superior exact-recovery performance against classical baselines, illustrating practical viability for learning tree-like Gaussian networks. The findings advance understanding of when structure can be learned efficiently from data under realistic assumptions and guide future work on extending to broader graphical models and non-Gaussian settings.

Abstract

We develop optimal algorithms for learning undirected Gaussian trees and directed Gaussian polytrees from data. We consider both problems of distribution learning (i.e. in KL distance) and structure learning (i.e. exact recovery). The first approach is based on the Chow-Liu algorithm, and learns an optimal tree-structured distribution efficiently. The second approach is a modification of the PC algorithm for polytrees that uses partial correlation as a conditional independence tester for constraint-based structure learning. We derive explicit finite-sample guarantees for both approaches, and show that both approaches are optimal by deriving matching lower bounds. Additionally, we conduct numerical experiments to compare the performance of various algorithms, providing further insights and empirical evidence.

Optimal estimation of Gaussian (poly)trees

TL;DR

This work provides a unified finite-sample analysis for learning Gaussian trees and polytrees, addressing both distribution learning (via KL distance) and structure learning (exact recovery). It introduces a Chow-Liu–type method for distribution learning and a PC-tree algorithm for polytree structure learning based on partial correlations, with explicit upper and matching lower bounds across non-realizable, realizable, and faithful scenarios. The results reveal phase transitions in sample complexity between distribution and structure learning and establish minimax optimality under strong-tree-faithfulness. Empirically, PC-Tree demonstrates superior exact-recovery performance against classical baselines, illustrating practical viability for learning tree-like Gaussian networks. The findings advance understanding of when structure can be learned efficiently from data under realistic assumptions and guide future work on extending to broader graphical models and non-Gaussian settings.

Abstract

We develop optimal algorithms for learning undirected Gaussian trees and directed Gaussian polytrees from data. We consider both problems of distribution learning (i.e. in KL distance) and structure learning (i.e. exact recovery). The first approach is based on the Chow-Liu algorithm, and learns an optimal tree-structured distribution efficiently. The second approach is a modification of the PC algorithm for polytrees that uses partial correlation as a conditional independence tester for constraint-based structure learning. We derive explicit finite-sample guarantees for both approaches, and show that both approaches are optimal by deriving matching lower bounds. Additionally, we conduct numerical experiments to compare the performance of various algorithms, providing further insights and empirical evidence.
Paper Structure (43 sections, 28 theorems, 114 equations, 14 figures, 3 algorithms)

This paper contains 43 sections, 28 theorems, 114 equations, 14 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Non-realizable Setting
  4. Realizable Setting
  5. Faithful Polytrees
  6. Other Related Work
  7. Learning Bayesian Networks
  8. Sample Complexity of Structure Learning
  9. Preliminaries and Tools
  10. Preliminary Notions
  11. Graphical Definitions
  12. Gaussian Bayesian Networks
  13. Faithfulness and Markov Equivalence Class
  14. Learning Tree-structured Gaussians
  15. Distribution Learning Upper Bounds
  16. ...and 28 more sections

Key Result

Theorem 3.1

Let $P$ be a Gaussian distribution. Given $n$ i.i.d. samples from $P$, for any $\varepsilon, \delta>0$, if $n \gtrsim\frac{d^2}{\varepsilon^2}\log \frac{d}{\delta}$, then $\widehat{T}$ returned by algo:chow-liu satisfies with probability at least $1-\delta$.

Figures (14)

  • Figure 1: Performance comparison for PC-Tree, Chow-Liu, PC and GES algorithm evaluated on SHD and PRR. The red, blue, green, purple lines are for PC-Tree, Chow-Liu, PC and GES respectively.
  • Figure 2: Four cases of $\ell$ to verify for $c$-strong Tree-faithfulness, indicated by the superscript of $X_\ell$. The first case is when $\ell=\emptyset$. The second, third and fourth are when $\ell$ is the ancestor of $j$, descendant of $j$ and descendant of $k$.
  • Figure 3: Construction for \ref{['lem:distlnvsstrucln']}.
  • Figure 4: The $\Omega(1/\varepsilon^2)$ bound in the non-realizable setting. The underlying graph is represented with solid lines, while the best estimated tree structure is depicted with dashed lines.
  • Figure 5: Realizable setting
  • ...and 9 more figures

Theorems & Definitions (51)

  • Definition 2.1: Faithfulness
  • Definition 2.2: Restricted faithfulness
  • Theorem 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Theorem 3.4
  • Definition 4.1: Tree-faithfulness
  • Definition 4.2: $c$-strong tree-faithfulness
  • Theorem 4.3
  • Theorem 4.4
  • ...and 41 more