CauScientist: Teaching LLMs to Respect Data for Causal Discovery

TL;DR

CauScientist addresses the core challenge of causal discovery by uniting LLM-driven hypothesis generation with a rigorous statistical verifier grounded in an Intervention-Aware BIC. The method relies on a hybrid initialization to start from a strong graph, followed by a collaborative loop where the LLM proposes atomic edits and a BIC-based verifier accepts only statistically beneficial changes, guided by an error-memory to prune the search. Empirical results show substantial improvements over purely data-driven baselines across multiple datasets, including up to a 53.8% gain in F1 and notable SHD reductions, while mitigating LLM unreliability as graphs scale. The approach demonstrates that semantic priors can be effectively leveraged when anchored by principled statistical constraints, offering a generalizable pathway for robust causal discovery in complex domains.

Abstract

Causal discovery is fundamental to scientific understanding and reliable decision-making. Existing approaches face critical limitations: purely data-driven methods suffer from statistical indistinguishability and modeling assumptions, while recent LLM-based methods either ignore statistical evidence or incorporate unverified priors that can mislead result. To this end, we propose CauScientist, a collaborative framework that synergizes LLMs as hypothesis-generating "data scientists" with probabilistic statistics as rigorous "verifiers". CauScientist employs hybrid initialization to select superior starting graphs, iteratively refines structures through LLM-proposed modifications validated by statistical criteria, and maintains error memory to guide efficient search space. Experiments demonstrate that CauScientist substantially outperforms purely data-driven baselines, achieving up to 53.8% F1 score improvement and enhancing recall from 35.0% to 100.0%. Notably, while standalone LLM performance degrades with graph complexity, CauScientist reduces structural hamming distance (SHD) by 44.0% compared to Qwen3-32B on 37-node graphs. Our project page is at https://github.com/OpenCausaLab/CauScientist.

Figures (10)

• Figure 1: Conceptual comparison of causal discovery methods. Data-driven models produce data-faithful answers but with inherent algorithm limitations. LLMs generate logically plausible answers, yet they often contradict statistical regularities. CauScientist combines the strengths of both methods, aligning semantic knowledge with data constraints.
• Figure 2: Pipeline of CauScientist. The framework operates in three stages: (1) Hybrid Initialization, where the initial graph $\mathcal{G}_0$ is selected from either a data-driven baseline or an LLM hypothesis based on the superior BIC score; (2) Collaborative Verification and Refinement, where the LLM proposes atomic modifications (e.g., adding an edge) that are rigorously evaluated by a statistical verifier for structural validity and BIC improvement; and (3) Iterative Optimization, where valid proposals update the graph state while rejected ones populate an error memory to prevent the LLM from repeating invalid moves.
• Figure 3: Optimization Trajectories of Qwen3-14B with AVICI as data-driven algorithm. LLM proposes reasonable edges operations during optimization loop (green circles), while BIC varifier successfully rejected operations with statistical inconsistency (red crosses). Note that for structure invalid errors (orange triangles), SHD is not computed. Therefore, we illustrate these marks on the x-axis.
• Figure 4: Score Validity on Alarm Dataset. We plot 100 perturbed graphs generated via 5 random walk trajectories. The X-axis represents structural error (SHD), and the Y-axis represents the intervention-aware BIC. The strong positive trend confirms that our scoring function effectively penalizes structural errors.
• Figure 5: Analysis of LLM optimization trajectories categorized by verification outcome. Larger models (e.g., Qwen3-32B) demonstrate superior reasoning capabilities, resulting in fewer structural violations and higher acceptance rates.