Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sparsifying transform priors in Gaussian graphical models

Marcus Gehrmann, Håkon Tjelmeland

TL;DR

This work tackles Bayesian structure learning in Gaussian graphical models by addressing the double intractability of the G-Wishart prior. It introduces Sparsifying Transform (ST) priors, where $Q = \text{PD}_G(\Sigma)$ with $\Sigma \sim \tilde{\pi}$, enabling MCMC that targets the exact posterior $\pi(Q,G|\mathbf{x})$ without approximating the normalizing constant $I_G(\delta,D)$. An auxiliary model with $(\Sigma,G)$ is developed, allowing standard Metropolis-Hastings updates and block-wise Sigma proposals via Schur complements, while preserving positive definiteness and avoiding intractable ratios. The method is demonstrated on a human gene expression dataset, showing convergence and distinct edge-posteriors compared to conventional G-Wishart-based methods, and highlighting the approach's flexibility and potential extensions for faster computation and symmetry considerations. Overall, ST priors provide a principled, adaptable alternative for Bayesian GGM structure learning with improved theoretical and practical properties.

Abstract

Bayesian methods constitute a popular approach for estimating the conditional independence structure in Gaussian graphical models, since they can quantify the uncertainty through the posterior distribution. Inference in this framework is typically carried out with Markov chain Monte Carlo (MCMC). However, the most widely used choice of prior distribution for the precision matrix, the so called G-Wishart distribution, suffers from an intractable normalizing constant, which gives rise to the problem of double intractability in the updating steps of the MCMC algorithm. In this article, we propose a new class of prior distributions for the precision matrix, termed ST priors, that allow for the construction of MCMC algorithms that do not suffer from double intractability issues. A realization from an ST prior distribution is obtained by applying a sparsifying transform on a matrix from a distribution with support in the set of all positive definite matrices. We carefully present the theory behind the construction of our proposed class of priors and also perform some numerical experiments, where we apply our methods on a human gene expression dataset. The results suggest that our proposed MCMC algorithm is able to converge and achieve acceptable mixing when applied on the real data.

Sparsifying transform priors in Gaussian graphical models

TL;DR

This work tackles Bayesian structure learning in Gaussian graphical models by addressing the double intractability of the G-Wishart prior. It introduces Sparsifying Transform (ST) priors, where with , enabling MCMC that targets the exact posterior without approximating the normalizing constant . An auxiliary model with is developed, allowing standard Metropolis-Hastings updates and block-wise Sigma proposals via Schur complements, while preserving positive definiteness and avoiding intractable ratios. The method is demonstrated on a human gene expression dataset, showing convergence and distinct edge-posteriors compared to conventional G-Wishart-based methods, and highlighting the approach's flexibility and potential extensions for faster computation and symmetry considerations. Overall, ST priors provide a principled, adaptable alternative for Bayesian GGM structure learning with improved theoretical and practical properties.

Abstract

Bayesian methods constitute a popular approach for estimating the conditional independence structure in Gaussian graphical models, since they can quantify the uncertainty through the posterior distribution. Inference in this framework is typically carried out with Markov chain Monte Carlo (MCMC). However, the most widely used choice of prior distribution for the precision matrix, the so called G-Wishart distribution, suffers from an intractable normalizing constant, which gives rise to the problem of double intractability in the updating steps of the MCMC algorithm. In this article, we propose a new class of prior distributions for the precision matrix, termed ST priors, that allow for the construction of MCMC algorithms that do not suffer from double intractability issues. A realization from an ST prior distribution is obtained by applying a sparsifying transform on a matrix from a distribution with support in the set of all positive definite matrices. We carefully present the theory behind the construction of our proposed class of priors and also perform some numerical experiments, where we apply our methods on a human gene expression dataset. The results suggest that our proposed MCMC algorithm is able to converge and achieve acceptable mixing when applied on the real data.
Paper Structure (20 sections, 1 theorem, 36 equations, 5 figures)

This paper contains 20 sections, 1 theorem, 36 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Graphs and notation
  4. Gaussian graphical models
  5. Prior distributions for the graph
  6. The G-Wishart distribution
  7. The Wishart and Inverse Wishart distributions
  8. A surjective map
  9. ST priors
  10. Definition of the class of distributions
  11. Using ST priors in Bayesian structure learning
  12. MCMC inference with ST priors
  13. Updating $\Sigma$
  14. Results
  15. Details about the data and normalization
  16. ...and 5 more sections

Key Result

Theorem 1

Grone1984 Let $G$ be a graph and $\mathcal{V}$ be the corresponding extended edge set. For any $\Sigma \in \mathbb{P}$, there is one and only one $Q \in \mathbb{P}(G)$ such that $\Sigma_{ij} = (Q^{-1})_{ij} \,\,\forall (i,j) \in \mathcal{V}$.

Figures (5)

  • Figure 1: Trace plots for posterior number of edges for the STMH algorithm when using each of the four different priors for the graph.
  • Figure 2: Histograms for posterior number of edges for the STMH algorithm when using each of the four different priors on the graph.
  • Figure 3: Estimated posterior edge probabilities for the SMTH algorithm when using each of the four different priors on the graph.
  • Figure 4: Fraction of edges with estimated posterior probability larger than or equal to $t$ as a function of $t$ for the SMTH algorithm with four different priors on the graph.
  • Figure 5: Histograms for number of edges for the BDgraph and WWA algorithms.

Theorems & Definitions (3)

  • Theorem 1
  • Definition 1
  • Definition 2