Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Guess2Graph: When and How Can Unreliable Experts Safely Boost Causal Discovery in Finite Samples?

Sujai Hiremath, Dominik Janzing, Philipp Faller, Patrick Blöbaum, Elke Kirschbaum, Shiva Prasad Kasiviswanathan, Kyra Gan

TL;DR

This work tackles finite-sample causal discovery when expert input (including LLMs) is fallible. It introduces Guess2Graph (G2G), a framework that guides the sequence of statistical tests rather than replacing them, preserving asymptotic correctness while enabling finite-sample improvements. Two instantiations are proposed: PC-Guess, which augments the PC algorithm, and gPC-Guess, a learning-augmented redesign that fully attains all three design criteria (consistency, monotone improvement, finite-sample robustness). Theoretical results show monotone gains with expert quality and end-to-end guarantees for the redesigned variant, and experiments demonstrate substantial accuracy and runtime benefits, including effective use with real-world LLM predictions. Collectively, the work provides a principled, robust approach to leveraging unreliable expert guidance in causal discovery with practical impact for high-stash, finite-sample settings.

Abstract

Causal discovery algorithms often perform poorly with limited samples. While integrating expert knowledge (including from LLMs) as constraints promises to improve performance, guarantees for existing methods require perfect predictions or uncertainty estimates, making them unreliable for practical use. We propose the Guess2Graph (G2G) framework, which uses expert guesses to guide the sequence of statistical tests rather than replacing them. This maintains statistical consistency while enabling performance improvements. We develop two instantiations of G2G: PC-Guess, which augments the PC algorithm, and gPC-Guess, a learning-augmented variant designed to better leverage high-quality expert input. Theoretically, both preserve correctness regardless of expert error, with gPC-Guess provably outperforming its non-augmented counterpart in finite samples when experts are "better than random." Empirically, both show monotonic improvement with expert accuracy, with gPC-Guess achieving significantly stronger gains.

From Guess2Graph: When and How Can Unreliable Experts Safely Boost Causal Discovery in Finite Samples?

TL;DR

This work tackles finite-sample causal discovery when expert input (including LLMs) is fallible. It introduces Guess2Graph (G2G), a framework that guides the sequence of statistical tests rather than replacing them, preserving asymptotic correctness while enabling finite-sample improvements. Two instantiations are proposed: PC-Guess, which augments the PC algorithm, and gPC-Guess, a learning-augmented redesign that fully attains all three design criteria (consistency, monotone improvement, finite-sample robustness). Theoretical results show monotone gains with expert quality and end-to-end guarantees for the redesigned variant, and experiments demonstrate substantial accuracy and runtime benefits, including effective use with real-world LLM predictions. Collectively, the work provides a principled, robust approach to leveraging unreliable expert guidance in causal discovery with practical impact for high-stash, finite-sample settings.

Abstract

Causal discovery algorithms often perform poorly with limited samples. While integrating expert knowledge (including from LLMs) as constraints promises to improve performance, guarantees for existing methods require perfect predictions or uncertainty estimates, making them unreliable for practical use. We propose the Guess2Graph (G2G) framework, which uses expert guesses to guide the sequence of statistical tests rather than replacing them. This maintains statistical consistency while enabling performance improvements. We develop two instantiations of G2G: PC-Guess, which augments the PC algorithm, and gPC-Guess, a learning-augmented variant designed to better leverage high-quality expert input. Theoretically, both preserve correctness regardless of expert error, with gPC-Guess provably outperforming its non-augmented counterpart in finite samples when experts are "better than random." Empirically, both show monotonic improvement with expert accuracy, with gPC-Guess achieving significantly stronger gains.
Paper Structure (75 sections, 25 theorems, 19 equations, 8 figures, 11 algorithms)

This paper contains 75 sections, 25 theorems, 19 equations, 8 figures, 11 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Setup and Guess2Graph
  4. Problem Statement and Design Criteria
  5. Guess2Graph Framework
  6. G2G in Constraint-Based Discovery
  7. A Tractable Metric for Analyzing Ordering Effects
  8. Guiding Edge Loop
  9. Guiding Edge Prune
  10. Expert Augmented Algorithms
  11. PC-Guess
  12. gPC-Guess
  13. Theoretical Guarantees
  14. Experiments
  15. Introduction
  16. ...and 60 more sections

Key Result

Lemma 4.1

For a fixed partial skeleton $\mathcal{C}$ and true DAG $\mathcal{G}^*$, let $\Phi_{\text{EL-G}}(p_{\psi})$ denote the perfect recovery probability when we sample an expert graph $\widehat{\mathcal{G}}$ from expert $\psi$ with accuracy $p_{\psi}$, draw $n$ samples from $\mathcal{G}^*$, and run EL-G.

Figures (8)

  • Figure 1: Guess2Graph uses expert graph predictions to reorder test sequences in causal discovery subroutines. Example above for constraint-based discovery.
  • Figure 2: Performance improvement of PC-Guess and gPC-Guess with expert guidance.
  • Figure I.1: Results for runtime when varying $p_\psi$.
  • Figure I.2: Method performance as $p_\psi$ increases in sparse graphs (ER1, $d=10$, $n=100$).
  • Figure I.3: Method performance across different values of $p_\psi$, in sparse graphs (ER1, $d=10$), as sample size is rapidly increased.
  • ...and 3 more figures

Theorems & Definitions (46)

  • Lemma 4.1: Monotonicity of Perfect Recovery in Expert Accuracy
  • Theorem 5.1: Asymptotic Correctness
  • Theorem 5.2: Performance of PC-Guess
  • Theorem 5.3: Performance of gPC-Guess
  • Remark 5.4
  • Definition B.1: Causal Markov Condition, spirtes_anytime_2001
  • Definition B.2: Acyclicity, spirtes_anytime_2001
  • Definition B.3: Faithfulness, spirtes_anytime_2001
  • Definition B.4: Causal Sufficiency, spirtes_anytime_2001
  • Definition B.5: d-Separation, spirtes_anytime_2001
  • ...and 36 more