Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Step-by-Step Causality: Transparent Causal Discovery with Multi-Agent Tree-Query and Adversarial Confidence Estimation

Ziyi Ding, Chenfei Ye-Hao, Zheyuan Wang, Xiao-Ping Zhang

TL;DR

This work tackles causal discovery under observational constraints by critiquing opaque LLM causal oracles and error-prone constraint-based methods. It proposes Tree-Query, a fixed, tree-structured, multi-expert reasoning framework with Adversarial Confidence Estimation to produce transparent, confidence-scored pairwise causal relations that assemble into a global graph. The authors provide asymptotic identifiability guarantees and demonstrate improved structural accuracy on data-free Mooij/UCI benchmarks, along with a case study on diet and weight showing effective confounder screening and stable causal directions. The framework offers data-free causal priors that can guide downstream data-driven discovery and is open-sourced to enable hybrid, interpretable causal reasoning pipelines with practitioners across domains.

Abstract

Causal discovery aims to recover ``what causes what'', but classical constraint-based methods (e.g., PC, FCI) suffer from error propagation, and recent LLM-based causal oracles often behave as opaque, confidence-free black boxes. This paper introduces Tree-Query, a tree-structured, multi-expert LLM framework that reduces pairwise causal discovery to a short sequence of queries about backdoor paths, (in)dependence, latent confounding, and causal direction, yielding interpretable judgments with robustness-aware confidence scores. Theoretical guarantees are provided for asymptotic identifiability of four pairwise relations. On data-free benchmarks derived from Mooij et al. and UCI causal graphs, Tree-Query improves structural metrics over direct LLM baselines, and a diet--weight case study illustrates confounder screening and stable, high-confidence causal conclusions. Tree-Query thus offers a principled way to obtain data-free causal priors from LLMs that can complement downstream data-driven causal discovery. Code is available at https://anonymous.4open.science/r/Repo-9B3E-4F96.

Step-by-Step Causality: Transparent Causal Discovery with Multi-Agent Tree-Query and Adversarial Confidence Estimation

TL;DR

This work tackles causal discovery under observational constraints by critiquing opaque LLM causal oracles and error-prone constraint-based methods. It proposes Tree-Query, a fixed, tree-structured, multi-expert reasoning framework with Adversarial Confidence Estimation to produce transparent, confidence-scored pairwise causal relations that assemble into a global graph. The authors provide asymptotic identifiability guarantees and demonstrate improved structural accuracy on data-free Mooij/UCI benchmarks, along with a case study on diet and weight showing effective confounder screening and stable causal directions. The framework offers data-free causal priors that can guide downstream data-driven discovery and is open-sourced to enable hybrid, interpretable causal reasoning pipelines with practitioners across domains.

Abstract

Causal discovery aims to recover ``what causes what'', but classical constraint-based methods (e.g., PC, FCI) suffer from error propagation, and recent LLM-based causal oracles often behave as opaque, confidence-free black boxes. This paper introduces Tree-Query, a tree-structured, multi-expert LLM framework that reduces pairwise causal discovery to a short sequence of queries about backdoor paths, (in)dependence, latent confounding, and causal direction, yielding interpretable judgments with robustness-aware confidence scores. Theoretical guarantees are provided for asymptotic identifiability of four pairwise relations. On data-free benchmarks derived from Mooij et al. and UCI causal graphs, Tree-Query improves structural metrics over direct LLM baselines, and a diet--weight case study illustrates confounder screening and stable, high-confidence causal conclusions. Tree-Query thus offers a principled way to obtain data-free causal priors from LLMs that can complement downstream data-driven causal discovery. Code is available at https://anonymous.4open.science/r/Repo-9B3E-4F96.
Paper Structure (45 sections, 5 theorems, 32 equations, 4 figures, 3 tables, 4 algorithms)

This paper contains 45 sections, 5 theorems, 32 equations, 4 figures, 3 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Problem Setup
  3. Notation
  4. Assumptions
  5. Problem Formulation
  6. Methodology
  7. Tree-Query Framework
  8. Design principles.
  9. Query types and MES--ACE interface.
  10. Overall Tree-Query procedure.
  11. Decision rule (aggregation layer).
  12. Comparison and summary.
  13. Multi-Expert System with Adversarial Confidence Estimation
  14. Multi-Expert System for Robust Causal Judgment
  15. Adversarial Confidence Estimator
  16. ...and 30 more sections

Key Result

Theorem 4.1

Under Assumptions a1--a4, consider any variable pair $(X_1, X_2)$ with its true causal relation Let Tree-Query consist of $M$ deterministic decision queries (decision stages), and assume that each query is answered by the Multi-Expert System using $m$ independent experts (as in Algorithm alg:mes_ace_combined in Appendix appendix:mes_ace), whose individual error probability satisfies $\alpha < Mo

Figures (4)

  • Figure 1: Tree-Query structure for inferring the causal relation between a variable pair $(X_1,X_2)$. Starting from an unknown relation (left), Tree-Query evaluates a fixed sequence of queries on the Tree-Query plane. The root issues a backdoor_path query: if no backdoor path is detected ($\hat{y}=0$), the procedure follows the left branch and successively asks independence, latent_confounder, and, if needed, causal_direction; if a backdoor path is detected ($\hat{y}=1$), the corresponding variables are conceptually blocked and the same sequence of queries is executed along the "after block" branch on the right. Each internal node outputs a local binary decision $\hat{y}$ with an associated confidence ("Confi."), and each root-to-leaf path ends in one of the candidate relations $\{X_1 \perp X_2,\; X_1 \leftrightarrow X_2,\; X_1 \rightarrow X_2,\; X_2 \rightarrow X_1\}$ shown in purple. Algorithm \ref{['alg:tree_checks']} evaluates all queries in this structure, and Algorithm \ref{['alg:tree_query_overall']} aggregates the leaf outcomes into a final relation and overall confidence (right). The implementation of each query node by the multi-expert and confidence-estimation modules is given in Fig. \ref{['MES_ACE']}.
  • Figure 2: Multi-Expert System (MES) with Adversarial Confidence Estimation (ACE) for a single Tree-Query node. Given a variable pair $(X_1,X_2)$ and a query type, the clinic agent selects $m$ experts from $K$ candidates and forwards the query. Each selected expert returns a binary conclusion $x^{(i)}_{\text{concl}}$ (step (1)). An adversarial agent then flips the conclusion and generates an adversarial argument, which is used to re-query the same expert configuration (steps (2)--(3)), producing perturbed conclusions ${x'}^{(i)}_{\text{concl}}$ (step (4)). ACE aggregates the original and perturbed conclusions across experts and adversarial personas to obtain a majority-vote label $\hat{y}$ and a robustness-based confidence score ("Confi."). This MES--ACE module implements each query node in the Tree-Query structure shown in Fig. \ref{['TreeQuery']}; the corresponding pseudocode for the combined MES-ACE module is given in Algorithm \ref{['alg:mes_ace_combined']} in Appendix \ref{['appendix:mes_ace']}.
  • Figure 3: Construction of the Latent benchmark by replacing hidden confounders with bidirected edges.
  • Figure 4: Confounder screening for the effect of diet on weight using Tree-Query. As additional variables are introduced, the confounding confidence (dashed arrows) decreases while the causal direction (diet $\rightarrow$ weight) remains stable.

Theorems & Definitions (8)

  • Theorem 4.1: Asymptotic Identifiability of Tree-Query
  • Proposition 4.2: Overall Causal Graph Identification Reliability
  • Proposition 4.3: Practical Feasibility Boundary
  • Lemma 4.5: Distributional shift
  • Proposition 4.6: Confidence as robustness against distributional shift
  • proof
  • proof
  • proof