Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Confounded Budgeted Causal Bandits

Fateme Jamshidi, Jalal Etesami, Negar Kiyavash

TL;DR

This work addresses budgeted causal multi-armed bandits with non-uniform intervention costs and hidden confounders in the underlying causal graph. It proposes two main algorithms—g-cumulative for cumulative regret and g-simple for simple regret—grounded in identifiability-based reward estimation from observational data and a cost-aware exploration strategy. The authors establish upper and lower bounds for both regret notions, and provide extensive experiments showing improvements over state-of-the-art baselines across general graphs and no-backdoor settings. The findings offer practical impact for budget-constrained experimentation where interventions carry varying costs and unobserved confounders are present, with applications in medicine, policy, and social science.

Abstract

We study the problem of learning 'good' interventions in a stochastic environment modeled by its underlying causal graph. Good interventions refer to interventions that maximize rewards. Specifically, we consider the setting of a pre-specified budget constraint, where interventions can have non-uniform costs. We show that this problem can be formulated as maximizing the expected reward for a stochastic multi-armed bandit with side information. We propose an algorithm to minimize the cumulative regret in general causal graphs. This algorithm trades off observations and interventions based on their costs to achieve the optimal reward. This algorithm generalizes the state-of-the-art methods by allowing non-uniform costs and hidden confounders in the causal graph. Furthermore, we develop an algorithm to minimize the simple regret in the budgeted setting with non-uniform costs and also general causal graphs. We provide theoretical guarantees, including both upper and lower bounds, as well as empirical evaluations of our algorithms. Our empirical results showcase that our algorithms outperform the state of the art.

Confounded Budgeted Causal Bandits

TL;DR

This work addresses budgeted causal multi-armed bandits with non-uniform intervention costs and hidden confounders in the underlying causal graph. It proposes two main algorithms—g-cumulative for cumulative regret and g-simple for simple regret—grounded in identifiability-based reward estimation from observational data and a cost-aware exploration strategy. The authors establish upper and lower bounds for both regret notions, and provide extensive experiments showing improvements over state-of-the-art baselines across general graphs and no-backdoor settings. The findings offer practical impact for budget-constrained experimentation where interventions carry varying costs and unobserved confounders are present, with applications in medicine, policy, and social science.

Abstract

We study the problem of learning 'good' interventions in a stochastic environment modeled by its underlying causal graph. Good interventions refer to interventions that maximize rewards. Specifically, we consider the setting of a pre-specified budget constraint, where interventions can have non-uniform costs. We show that this problem can be formulated as maximizing the expected reward for a stochastic multi-armed bandit with side information. We propose an algorithm to minimize the cumulative regret in general causal graphs. This algorithm trades off observations and interventions based on their costs to achieve the optimal reward. This algorithm generalizes the state-of-the-art methods by allowing non-uniform costs and hidden confounders in the causal graph. Furthermore, we develop an algorithm to minimize the simple regret in the budgeted setting with non-uniform costs and also general causal graphs. We provide theoretical guarantees, including both upper and lower bounds, as well as empirical evaluations of our algorithms. Our empirical results showcase that our algorithms outperform the state of the art.
Paper Structure (26 sections, 20 theorems, 109 equations, 24 figures, 5 algorithms)

This paper contains 26 sections, 20 theorems, 109 equations, 24 figures, 5 algorithms.

Table of Contents

  1. Introduction
  2. Contribution:
  3. Related Work
  4. Preliminaries
  5. Cumulative Regret in General Graphs
  6. Estimation and update steps
  7. Simple Regret in General Graphs
  8. Lower Bounds
  9. Simple regret:
  10. Cumulative regret:
  11. Experiments
  12. Cumulative Regret in General Graphs
  13. Simple Regret in No-backdoor Graphs
  14. Simple Regret in General Graphs
  15. Conclusion
  16. ...and 11 more sections

Key Result

Lemma 3

$\hat{\mu}_{i,x}^t$ in eq: estimate mu and $\hat{\mu}_{0}^{t}$ in eq: estimate mu0 are unbiased estimators of $\mu_{i,x}$ and $\mu_0$.

Figures (24)

  • Figure 1: An ADMAG $\mathcal{G}$ over $\textbf{V}=\{X_1,...,X_5\}$. Bidirected edges are represented by dashed edges.
  • Figure 2: Budgeted Cumulative Regret in General Graphs
  • Figure 3: Budgeted Simple Regret in General Graphs
  • Figure 4: Cumulative regret on a general graph with $N=6$.
  • Figure 5: Simple regret on the parallel graph with $N=50$.
  • ...and 19 more figures

Theorems & Definitions (31)

  • Definition 1: C-component tian2002general
  • Definition 2: Identifiability tian2002general
  • Remark 3
  • Lemma 3
  • Theorem 4
  • Remark 4
  • Theorem 5
  • Remark 6
  • Remark 6
  • Definition 7: Directed Path
  • ...and 21 more