Ice Cream Doesn't Cause Drowning: Benchmarking LLMs Against Statistical Pitfalls in Causal Inference

TL;DR

The paper tackles the problem of evaluating LLMs for statistical causal inference, showing that accuracy alone is insufficient due to vulnerabilities to common pitfalls. It proposes CausalPitfalls, a benchmark with six pitfall categories, 15 challenges, 75 questions, and 75 datasets, and uses two evaluation protocols—direct prompting and code-assisted prompting—along with a causal reliability metric defined as the average normalized score across challenges, $\text{CR} = \frac{1}{K} \sum_{k=1}^{K} \frac{\text{score}_k}{\text{max\_score}_k} \times 100\%$. Results reveal substantial reliability gaps across models; code-assisted prompting yields robust gains but even the best model attains only about $44.6\%$ causal reliability on average, with execution errors concentrated in mediation and external validity tasks. The work provides a practical framework for measuring and improving trustworthy causal reasoning in AI, and outlines future directions including instrumental-variable approaches, latent confounding, and cross-domain transportability to broaden applicability in policy and medicine.

Abstract

Reliable causal inference is essential for making decisions in high-stakes areas like medicine, economics, and public policy. However, it remains unclear whether large language models (LLMs) can handle rigorous and trustworthy statistical causal inference. Current benchmarks usually involve simplified tasks. For example, these tasks might only ask LLMs to identify semantic causal relationships or draw conclusions directly from raw data. As a result, models may overlook important statistical pitfalls, such as Simpson's paradox or selection bias. This oversight limits the applicability of LLMs in the real world. To address these limitations, we propose CausalPitfalls, a comprehensive benchmark designed to rigorously evaluate the capability of LLMs in overcoming common causal inference pitfalls. Our benchmark features structured challenges across multiple difficulty levels, each paired with grading rubrics. This approach allows us to quantitatively measure both causal reasoning capabilities and the reliability of LLMs' responses. We evaluate models using two protocols: (1) direct prompting, which assesses intrinsic causal reasoning, and (2) code-assisted prompting, where models generate executable code for explicit statistical analysis. Additionally, we validate the effectiveness of this judge by comparing its scoring with assessments from human experts. Our results reveal significant limitations in current LLMs when performing statistical causal inference. The CausalPitfalls benchmark provides essential guidance and quantitative metrics to advance the development of trustworthy causal reasoning systems.

Figures (4)

• Figure 1: Overall Message: Our results reveal a clear reliability gap in causal inference when LLMs rely only on direct prompting, with all models struggling most on mediation and external validity questions. Introducing code assisted prompting leads to substantial gains across every task and brings all models closer together in performance. This shows that executable analysis is essential for large language models to handle complex statistical challenges and deliver trustworthy causal conclusions.
• Figure 2: High-level overview of the CausalPitfalls benchmark. (a) An illustrative real-world pitfall (Simpson’s paradox): when data on treatment consumption and recovery are pooled (top), a naıve analysis finds a positive effect ("Helpful!"), but stratifying by age reveals a negative effect within both younger and older subgroups ("Harmful!"). (b) Benchmark workflow: LLMs are evaluated under two protocols: (1) Direct Prompting on raw data, assessing intrinsic causal reasoning, and (2) Code-Assisted Prompting on sampled data, assessing computationally grounded inference. In both cases, model answers are automatically scored against a hidden grading rubric by an independent grader to quantify each model’s causal reliability.
• Figure 3: Causal DAG illustrating how beverage consumption, health awareness, and lifestyle affect health outcomes. The beverage's brand name ("HealthPlus" or "UltraSugar") does not causally influence outcomes.
• Figure 4: Code execution failure rates (%) in code-assisted prompting protocol across causal inference challenges and question difficulty. Failure rate is defined as the percentage of code-generation attempts that either raise execution errors or produce invalid analytical outputs, computed only for the code-assisted prompting protocol. (a) Average failure rate for each of the six causal-inference pitfall categories. (b) Average failure rate by question difficulty level, increasing from very easy through very hard tasks.