Covariate Dependent Mixture of Bayesian Networks

TL;DR

Covariate-Dependent Mixture of Bayesian Networks addresses BN structure learning under population heterogeneity by modeling $p(\boldsymbol{y}|\boldsymbol{x})$ as a mixture of DAG-based BNs whose component weights $\pi_k(\boldsymbol{x})$ depend on covariates via multinomial logistic gates. Each component is a Gaussian linear-regression BN with priors on graphs and parameters, enabling a closed-form marginal likelihood (BGe) for inference. The joint posterior over $\mathcal{Z},\mathcal{G},\tilde{\boldsymbol{\beta}}$ is sampled with an MCMC scheme (block Gibbs, Polson data augmentation, Partition MCMC), and the number of mixtures $K$ is selected via cross-validated Log Marginal Posterior Predictive Density. Empirical results on synthetic data show accurate recovery of mixture components when $K$ is known and correctly identified, while a youth mental health case study demonstrates distinct subgroups with differing causal structures, supporting personalized interventions and improved interpretability in heterogeneous populations.

Abstract

Learning the structure of Bayesian networks from data provides insights into underlying processes and the causal relationships that generate the data, but its usefulness depends on the homogeneity of the data population, a condition often violated in real-world applications. In such cases, using a single network structure for inference can be misleading, as it may not capture sub-population differences. To address this, we propose a novel approach of modelling a mixture of Bayesian networks where component probabilities depend on individual characteristics. Our method identifies both network structures and demographic predictors of sub-population membership, aiding personalised interventions. We evaluate our method through simulations and a youth mental health case study, demonstrating its potential to improve tailored interventions in health, education, and social policy.

Figures (6)

• Figure 1: MSHD between estimated graphs and ground truth, for varying number of true mixture components, $\mathcal{K}$ as sub-figures, number of observations per mixture,$N$, as the horizontal axis; network sparsity $S$, cluster density in non-modifiables space $\mathcal{C}$ as the combinations in the legend.
• Figure 2: Boxplots of the log marginal posterior predictive density LMPPD of 10 realizations from the model for $\mathcal{K}\in\{1,\ldots,4\}$, $\mathcal{N}=500$$S=\hbox{Low}$, estimated using $K\in\{1,\ldots,5\}$.
• Figure 3: MAP Graphs for each cluster and total number of clusters
• Figure 4: Total log marginal predictive probability density (LMPPD) for the test data over a varying number of mixture components $K$. Each box summarises the scores over 10 different runs of the experiments with different starting DAGs.
• Figure 5: WAIC scores over in-sample data for the real-world case study in \ref{['sec:mental_health']}. For each mixture component $K$, 10 MCMC runs were conducted, each initialized with a different starting DAG.