Multi-Objective Multi-Fidelity Bayesian Optimization with Causal Priors

TL;DR

This work tackles efficient optimization of expensive, black-box systems when multiple objectives must be balanced and multiple fidelity sources are available. It introduces RESCUE, a framework that learns a structural causal model and builds a MF-CGP surrogate to encode interventional effects across inputs, fidelities, and constraints. A novel Causal Hypervolume Knowledge-Gradient (C-HVKG) acquisition then selects input-fidelity pairs by trading off expected Pareto-improvement against cost, with theoretical guarantees that cumulative hypervolume regret remains bounded even with unreliable auxiliary sources. Empirically, RESCUE achieves superior sample efficiency and constraint satisfaction across synthetic benchmarks and real-world problems in robotics, AutoML, and healthcare, outperforming both standard MFBO and non-MF MOBO baselines. The approach promises improved optimization efficiency and interpretability in complex, high-stakes systems by leveraging causal structure to guide cross-fidelity information gathering.

Abstract

Multi-fidelity Bayesian optimization (MFBO) accelerates the search for the global optimum of black-box functions by integrating inexpensive, low-fidelity approximations. The central task of an MFBO policy is to balance the cost-efficiency of low-fidelity proxies against their reduced accuracy to ensure effective progression toward the high-fidelity optimum. Existing MFBO methods primarily capture associational dependencies between inputs, fidelities, and objectives, rather than causal mechanisms, and can perform poorly when lower-fidelity proxies are poorly aligned with the target fidelity. We propose RESCUE (REducing Sampling cost with Causal Understanding and Estimation), a multi-objective MFBO method that incorporates causal calculus to systematically address this challenge. RESCUE learns a structural causal model capturing causal relationships between inputs, fidelities, and objectives, and uses it to construct a probabilistic multi-fidelity (MF) surrogate that encodes intervention effects. Exploiting the causal structure, we introduce a causal hypervolume knowledge-gradient acquisition strategy to select input-fidelity pairs that balance expected multi-objective improvement and cost. We show that RESCUE improves sample efficiency over state-of-the-art MF optimization methods on synthetic and real-world problems in robotics, machine learning (AutoML), and healthcare.

Figures (14)

• Figure 1: Real-robot environments with and without obstacles (left and middle, respectively), and a Gazebo simulation (right).
• Figure 2: Cumulative intervention cost vs. best log HV regret, where “best’’ denotes the minimum regret achieved up to that cost.
• Figure 3: Fidelity queries and iteration counts under a fixed budget. For each problem, the top panel reports the number of iterations achieved by each method, and the bottom panel shows the empirical distribution of queried fidelities. Each violin shows the fidelity distribution with a vertical min–max range line and a mean marker.
• Figure 3: Robot navigation task surrogate model performance.
• Figure 4: Constraint violation rate percentage.

Theorems & Definitions (13)

• Definition 4.1
• Definition 5.1
• Lemma 5.6: Misspecified MF-CGP confidence bound
• Lemma 5.7: HV approximation bound
• Lemma 5.8: One-step acquisition comparison
• Theorem 5.9: RESCUE cumulative HV regret
• Corollary 5.10: Unreliable auxiliary source robustness
• proof
• Remark 1.1
• proof