Extending Multi-Source Bayesian Optimization With Causality Principles

TL;DR

MSCBO tackles multi-source Bayesian optimization by integrating causal structure (CBO) and multi-source information (MSBO) within a single framework. It uses per-source Gaussian processes and a cost-sensitive Knowledge Gradient to select interventions efficiently, while leveraging Posterior Optimal Minimal Intervention Sets to prune the search space and an epsilon-greedy policy to balance exploration and exploitation under budget. The approach is validated on PSA and E. coli networks, showing comparable or improved optima with reduced intervention cost and greater robustness to noise, particularly in larger or more complex networks. The work demonstrates practical potential for cost-efficient, scalable causal optimization in real-world domains such as clinical trials and policy planning, and outlines avenues for reducing computational overhead and extending to discrete networks.

Abstract

Multi-Source Bayesian Optimization (MSBO) serves as a variant of the traditional Bayesian Optimization (BO) framework applicable to situations involving optimization of an objective black-box function over multiple information sources such as simulations, surrogate models, or real-world experiments. However, traditional MSBO assumes the input variables of the objective function to be independent and identically distributed, limiting its effectiveness in scenarios where causal information is available and interventions can be performed, such as clinical trials or policy-making. In the single-source domain, Causal Bayesian Optimization (CBO) extends standard BO with the principles of causality, enabling better modeling of variable dependencies. This leads to more accurate optimization, improved decision-making, and more efficient use of low-cost information sources. In this article, we propose a principled integration of the MSBO and CBO methodologies in the multi-source domain, leveraging the strengths of both to enhance optimization efficiency and reduce computational complexity in higher-dimensional problems. We present the theoretical foundations of both Causal and Multi-Source Bayesian Optimization, and demonstrate how their synergy informs our Multi-Source Causal Bayesian Optimization (MSCBO) algorithm. We compare the performance of MSCBO against its foundational counterparts for both synthetic and real-world datasets with varying levels of noise, highlighting the robustness and applicability of MSCBO. Based on our findings, we conclude that integrating MSBO with the causality principles of CBO facilitates dimensionality reduction and lowers operational costs, ultimately improving convergence speed, performance, and scalability.

Figures (13)

• Figure 1: Example DAG for cardiovascular disease treatment. Gray nodes represent variables that can be intervened upon. The output variable Cardiovascular Disease is denoted with a thick-dashed node.
• Figure 2: A visualization of the theoretical comparison between each of the optimization methods as provided in Section \ref{['sec:integration']}. We assume the found optimum over each iteration to be identical for the three algorithms.
• Figure 3: MSCBO Conceptual Diagram: As its input the algorithm takes information from multiple sources, in the form of DAG's and their associated observation values, and an optimization query (represented by the question mark). The POMIS-algorithm serves as a subroutine to prune exploration sets. Based on these DAG's, the algorithm proposes an intervention set value assignment that maximizes or minimizes the posterior value.
• Figure 4: Causal DAG depicting the PSA scenario. Gray nodes represent variables that can be intervened upon, and dashed nodes represent non-manipulative variables, respectively. The output variable PSA is denoted with a thick-dashed node.
• Figure 5: Optima over cost for the three different implementations within the PSA graph. The true optimum is represented by the red line in the graph.