Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Causal Preference Elicitation

Edwin V. Bonilla, He Zhao, Daniel M. Steinberg

TL;DR

CaPE formulates causal discovery as a Bayesian active-learning problem where an expert provides noisy, local judgments about edge existence and orientation. By modeling expert feedback with a hierarchical three-way likelihood and updating a particle-based posterior over DAGs, the method actively selects edge queries via an expected information gain criterion to accelerate posterior concentration. Across synthetic, Sachs, and CausalBench benchmarks, CaPE achieves faster uncertainty reduction and improved directed-edge recovery under tight query budgets, and benefits further from strong observational priors such as DAG-GFlowNet. The framework is modular, scalable, and adaptable to richer expert models, offering a practical route to reliable causal structure learning in settings with limited interventional data. The integration of BALD-style acquisition with sequential Monte Carlo over DAGs provides a principled approach to leveraging human expertise in causal inference.

Abstract

We propose causal preference elicitation, a Bayesian framework for expert-in-the-loop causal discovery that actively queries local edge relations to concentrate a posterior over directed acyclic graphs (DAGs). From any black-box observational posterior, we model noisy expert judgments with a three-way likelihood over edge existence and direction. Posterior inference uses a flexible particle approximation, and queries are selected by an efficient expected information gain criterion on the expert's categorical response. Experiments on synthetic graphs, protein signaling data, and a human gene perturbation benchmark show faster posterior concentration and improved recovery of directed effects under tight query budgets.

Causal Preference Elicitation

TL;DR

CaPE formulates causal discovery as a Bayesian active-learning problem where an expert provides noisy, local judgments about edge existence and orientation. By modeling expert feedback with a hierarchical three-way likelihood and updating a particle-based posterior over DAGs, the method actively selects edge queries via an expected information gain criterion to accelerate posterior concentration. Across synthetic, Sachs, and CausalBench benchmarks, CaPE achieves faster uncertainty reduction and improved directed-edge recovery under tight query budgets, and benefits further from strong observational priors such as DAG-GFlowNet. The framework is modular, scalable, and adaptable to richer expert models, offering a practical route to reliable causal structure learning in settings with limited interventional data. The integration of BALD-style acquisition with sequential Monte Carlo over DAGs provides a principled approach to leveraging human expertise in causal inference.

Abstract

We propose causal preference elicitation, a Bayesian framework for expert-in-the-loop causal discovery that actively queries local edge relations to concentrate a posterior over directed acyclic graphs (DAGs). From any black-box observational posterior, we model noisy expert judgments with a three-way likelihood over edge existence and direction. Posterior inference uses a flexible particle approximation, and queries are selected by an efficient expected information gain criterion on the expert's categorical response. Experiments on synthetic graphs, protein signaling data, and a human gene perturbation benchmark show faster posterior concentration and improved recovery of directed effects under tight query budgets.
Paper Structure (94 sections, 10 theorems, 98 equations, 8 figures, 3 tables, 6 algorithms)

This paper contains 94 sections, 10 theorems, 98 equations, 8 figures, 3 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Setup
  4. Expert Likelihood Model
  5. Local Direction Scores
  6. Hierarchical Logistic Expert Model
  7. Edge-existence probability.
  8. Direction probability conditional on existence.
  9. Three-way Likelihood for $Y_{ij}$
  10. Posterior Representation and Update
  11. Bayesian Update
  12. Particle Representation
  13. Weight update.
  14. Effective Sample Size and Resampling
  15. Particle Rejuvenation
  16. ...and 79 more sections

Key Result

Proposition 7.1

Fix $(i,j)$ and let $q_t(W)$ denote the current posterior. For each possible label $y\in\{0,1,2\}$, define the updated posterior after hypothetically observing $Y_{ij}=y$ by where $\widehat{p}_t^{\,ij}(y)$ is the posterior predictive probability of $Y_{ij}=y$. Then the EIG can be written as

Figures (8)

  • Figure 1: Synthetic-data results comparing expert query strategies. Average predictive entropy (Entropy $\downarrow$), structural Hamming distance (SHD $\downarrow$) to the ground-truth graph, and expected true class probability (ETCP $\uparrow$). The means $\pm$ one standard deviations (as shaded areas) across 10 replications are reported.
  • Figure 2: Results on Sachs observational-only benchmark: Average predictive entropy (Entropy $\downarrow$), structural Hamming distance (SHD $\downarrow$) to the ground-truth graph, and expected true class probability (ETCP $\uparrow$). The means $\pm$ one standard deviations (as shaded areas) across 10 replications are reported.
  • Figure 3: Posterior edge probabilities on Sachs (observational-only): (left) initial observational posterior $q_0$, (middle) posterior after $T{=}40$ EIG-selected expert queries, (right) difference between final and initial posteriors. Targeted querying sharpens posterior beliefs over directed edges.
  • Figure 4: Results on the CausalBench K562 50-gene benchmark. We report directed AUPRC (left) and Top-$K$ precision (right, with $K$ equal to the number of oracle edges) as a function of the number of expert queries. Shaded regions indicate $\pm 1$ standard deviation over random seeds.
  • Figure 5: Results on Sachs observational-only benchmark with a DAG-GFN prior: Average predictive entropy (Entropy $\downarrow$), structural Hamming distance (SHD $\downarrow$) to the ground-truth graph, and expected true class probability (ETCP $\uparrow$). Shaded regions indicate $\pm 1$ standard deviation over random seeds.
  • ...and 3 more figures

Theorems & Definitions (19)

  • Proposition 7.1: EIG and expected KL contraction
  • Theorem 7.1: Identifiability under non-adversarial expert feedback
  • Lemma 4.1: BALD decomposition for $Y_{ij}$
  • proof
  • Proposition 4.1: Restatement of \ref{['prop:eig-kl']}, EIG and expected KL contraction
  • proof
  • Remark 4.1: Query selection as maximizing expected posterior change
  • Lemma 4.2: Mixture form of $I_t(W;Y_{ij})$
  • proof
  • Remark 4.2: EIG as disagreement in the simplex
  • ...and 9 more