Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Bayesian Take on Gaussian Process Networks

Enrico Giudice, Jack Kuipers, Giusi Moffa

TL;DR

The paper tackles learning directed graphical models with continuous, potentially nonlinear dependencies by embedding Gaussian Process Networks (GPNs) into a fully Bayesian structure inference framework. It develops a sampling scheme over graph space using order and partition MCMC, augmented with priors on GP hyperparameters and efficient marginal likelihood estimation via Laplace approximation and bridge sampling, enabling posterior inference over graphs and features. Through simulations and a protein signaling application (Sachs dataset), the authors show improved structure recovery in nonlinear regimes and robust uncertainty quantification, while maintaining competitive performance in linear settings. The approach advances principled, uncertainty-aware structure learning for complex, real-valued domains where parametric assumptions are insufficient, with practical impact in systems biology and beyond.

Abstract

Gaussian Process Networks (GPNs) are a class of directed graphical models which employ Gaussian processes as priors for the conditional expectation of each variable given its parents in the network. The model allows the description of continuous joint distributions in a compact but flexible manner with minimal parametric assumptions on the dependencies between variables. Bayesian structure learning of GPNs requires computing the posterior over graphs of the network and is computationally infeasible even in low dimensions. This work implements Monte Carlo and Markov Chain Monte Carlo methods to sample from the posterior distribution of network structures. As such, the approach follows the Bayesian paradigm, comparing models via their marginal likelihood and computing the posterior probability of the GPN features. Simulation studies show that our method outperforms state-of-the-art algorithms in recovering the graphical structure of the network and provides an accurate approximation of its posterior distribution.

A Bayesian Take on Gaussian Process Networks

TL;DR

The paper tackles learning directed graphical models with continuous, potentially nonlinear dependencies by embedding Gaussian Process Networks (GPNs) into a fully Bayesian structure inference framework. It develops a sampling scheme over graph space using order and partition MCMC, augmented with priors on GP hyperparameters and efficient marginal likelihood estimation via Laplace approximation and bridge sampling, enabling posterior inference over graphs and features. Through simulations and a protein signaling application (Sachs dataset), the authors show improved structure recovery in nonlinear regimes and robust uncertainty quantification, while maintaining competitive performance in linear settings. The approach advances principled, uncertainty-aware structure learning for complex, real-valued domains where parametric assumptions are insufficient, with practical impact in systems biology and beyond.

Abstract

Gaussian Process Networks (GPNs) are a class of directed graphical models which employ Gaussian processes as priors for the conditional expectation of each variable given its parents in the network. The model allows the description of continuous joint distributions in a compact but flexible manner with minimal parametric assumptions on the dependencies between variables. Bayesian structure learning of GPNs requires computing the posterior over graphs of the network and is computationally infeasible even in low dimensions. This work implements Monte Carlo and Markov Chain Monte Carlo methods to sample from the posterior distribution of network structures. As such, the approach follows the Bayesian paradigm, comparing models via their marginal likelihood and computing the posterior probability of the GPN features. Simulation studies show that our method outperforms state-of-the-art algorithms in recovering the graphical structure of the network and provides an accurate approximation of its posterior distribution.
Paper Structure (16 sections, 21 equations, 12 figures, 1 table, 1 algorithm)

This paper contains 16 sections, 21 equations, 12 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Bayesian Structure Inference
  4. Gaussian Processes
  5. Gaussian Process Networks
  6. Bayesian Structure Inference for GPNs
  7. Implementation Details
  8. Score Equivalence
  9. Experimental Results
  10. Choice of Priors
  11. Results
  12. Application on Protein Signaling Networks
  13. Conclusions
  14. Supplementary Material
  15. Measuring Score Equivalence
  16. ...and 1 more sections

Figures (12)

  • Figure 1: Distribution of $\mathbb{E}\mathrm{-SHD}$ values for all the different algorithms. $\lambda=0$ corresponds to linear-Gaussian data while higher values increase the degree of non-linearity of the relations among variables.
  • Figure 2: Distribution of run-times for all the different algorithms. $\lambda=0$ corresponds to linear-Gaussian data while higher values increase the degree of non-linearity of the relations among variables.
  • Figure 3: Left: reverse K-L divergence between the true posterior and the BGe posterior (green), the Laplace approximate posterior (blue) and the posterior obtained via importance sampling (red) as a function of the number of sampled DAGs. Right: the true posterior (gray) together with the BGe posterior (green), the Laplace approximate posterior (blue) and the posterior obtained via importance sampling (red). The majority of DAGs have a very low true posterior probability and are therefore never sampled by the MCMC algorithms (see inset).
  • Figure A.1: The median difference in GP log score between the forward and backward model, with $0.1$- and $0.9$-quantiles.
  • Figure A.2: Average FPRp and TPR values for a selection of the different algorithms. $\lambda=0$ corresponds to linear-Gaussian data while higher values increase the degree of non-linearity of the relations among variables.
  • ...and 7 more figures