Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayes

Mizu Nishikawa-Toomey, Tristan Deleu, Jithendaraa Subramanian, Yoshua Bengio, Laurent Charlin

TL;DR

The paper addresses uncertainty quantification in causal structure learning by jointly inferring DAGs and causal mechanisms. It combines Variational Bayes with a GFlowNet (VBG) to model the posterior $P(G,\theta|\mathcal{D})$, enabling sampling of acyclic graphs while estimating mechanism parameters under a linear-Gaussian model. Empirically, VBG is competitive with baseline Bayesian methods on synthetic graphs and real protein-signalling data, and it offers acyclicity guarantees and a clear path to extending to non-linear mechanisms. This approach advances practical causal inference by providing a scalable, uncertainty-aware framework for joint graph and mechanism learning with potential for active intervention design.

Abstract

Bayesian causal structure learning aims to learn a posterior distribution over directed acyclic graphs (DAGs), and the mechanisms that define the relationship between parent and child variables. By taking a Bayesian approach, it is possible to reason about the uncertainty of the causal model. The notion of modelling the uncertainty over models is particularly crucial for causal structure learning since the model could be unidentifiable when given only a finite amount of observational data. In this paper, we introduce a novel method to jointly learn the structure and mechanisms of the causal model using Variational Bayes, which we call Variational Bayes-DAG-GFlowNet (VBG). We extend the method of Bayesian causal structure learning using GFlowNets to learn not only the posterior distribution over the structure, but also the parameters of a linear-Gaussian model. Our results on simulated data suggest that VBG is competitive against several baselines in modelling the posterior over DAGs and mechanisms, while offering several advantages over existing methods, including the guarantee to sample acyclic graphs, and the flexibility to generalize to non-linear causal mechanisms.

Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayes

TL;DR

The paper addresses uncertainty quantification in causal structure learning by jointly inferring DAGs and causal mechanisms. It combines Variational Bayes with a GFlowNet (VBG) to model the posterior , enabling sampling of acyclic graphs while estimating mechanism parameters under a linear-Gaussian model. Empirically, VBG is competitive with baseline Bayesian methods on synthetic graphs and real protein-signalling data, and it offers acyclicity guarantees and a clear path to extending to non-linear mechanisms. This approach advances practical causal inference by providing a scalable, uncertainty-aware framework for joint graph and mechanism learning with potential for active intervention design.

Abstract

Bayesian causal structure learning aims to learn a posterior distribution over directed acyclic graphs (DAGs), and the mechanisms that define the relationship between parent and child variables. By taking a Bayesian approach, it is possible to reason about the uncertainty of the causal model. The notion of modelling the uncertainty over models is particularly crucial for causal structure learning since the model could be unidentifiable when given only a finite amount of observational data. In this paper, we introduce a novel method to jointly learn the structure and mechanisms of the causal model using Variational Bayes, which we call Variational Bayes-DAG-GFlowNet (VBG). We extend the method of Bayesian causal structure learning using GFlowNets to learn not only the posterior distribution over the structure, but also the parameters of a linear-Gaussian model. Our results on simulated data suggest that VBG is competitive against several baselines in modelling the posterior over DAGs and mechanisms, while offering several advantages over existing methods, including the guarantee to sample acyclic graphs, and the flexibility to generalize to non-linear causal mechanisms.
Paper Structure (31 sections, 29 equations, 11 figures, 2 tables)

This paper contains 31 sections, 29 equations, 11 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Causal structure learning
  4. Bayesian causal structure learning
  5. Background
  6. Causal modelling
  7. Generative Flow Networks
  8. GFlowNets for causal structure learning
  9. Variational Bayes-DAG-GFlowNet
  10. Modelling the distribution over graphs with a GFlowNet
  11. Updating the distribution over mechanism parameters
  12. Assumptions and limitations
  13. Experiments
  14. Synthetic data generation
  15. Baselines
  16. ...and 16 more sections

Figures (11)

  • Figure 1: MSE of Edge, path and Markov features of the true posterior and the estimated posterior for 5 node Erdos-Renyi graphs (lower the better).
  • Figure 2: $\mathbb{E}$-SHD (lower the better) inferring Erdos Renyi graphs with differing number of nodes. Box plots correspond to the median and 25th and 75th percentiles.
  • Figure 3: AUROC (higher the better) inferring Erdos Renyi graphs with differing number of nodes. Box plots correspond to the median and 25th and 75th percentiles.
  • Figure 4: MSE $\theta$ (lower the better) inferring Erdos Renyi graphs with differing number of nodes. Box plots correspond to the median and 25th and 75th percentiles.
  • Figure 5: Negative log-likelihood of held-out data for Erdos-Renyi graphs with differing number of nodes (lower the better). Metropolis-Hasting (MH) results had extremely large values so were omitted here but can be seen in \ref{['app:results-tables']}. Results are averaged over 20 graphs
  • ...and 6 more figures