On the Eligibility of LLMs for Counterfactual Reasoning: A Decompositional Study

TL;DR

This work introduces a decompositional framework for evaluating LLM counterfactual reasoning by separating causality construction from counterfactual reasoning and testing four stages: variable identification, DAG construction, intervention identification, and outcome reasoning, across 11 multimodal datasets. It demonstrates that LLMs struggle most with implicit mediators and downstream reasoning, while graph construction is comparatively robust when prior steps are provided. To address bottlenecks, the authors propose tool-augmented variable identification and advanced elicitation strategies (CoT, CoT-SC, ToT), yielding notable improvements in explicit-variable identification and, to a more nuanced extent, in implicit-variable reasoning. The benchmark and findings offer a structured path to enhance the reliability of LLM-based reasoning systems and inform future elicitation and multimodal reasoning methods, with implications for interpretable AI and causality-aware NLP.

Abstract

Counterfactual reasoning has emerged as a crucial technique for generalizing the reasoning capabilities of large language models (LLMs). By generating and analyzing counterfactual scenarios, researchers can assess the adaptability and reliability of model decision-making. Although prior work has shown that LLMs often struggle with counterfactual reasoning, it remains unclear which factors most significantly impede their performance across different tasks and modalities. In this paper, we propose a decompositional strategy that breaks down the counterfactual generation from causality construction to the reasoning over counterfactual interventions. To support decompositional analysis, we investigate 11 datasets spanning diverse tasks, including natural language understanding, mathematics, programming, and vision-language tasks. Through extensive evaluations, we characterize LLM behavior across each decompositional stage and identify how modality type and intermediate reasoning influence performance. By establishing a structured framework for analyzing counterfactual reasoning, this work contributes to the development of more reliable LLM-based reasoning systems and informs future elicitation strategies.

Figures (6)

• Figure 1: A workflow and illustrative example that decomposes LLM-based counterfactual reasoning into four stages: (1) identifying causal variables (e.g., whether web recommendation is shown), (2) constructing the causal graph (e.g., browsing history $\rightarrow$ a recommendation is shown), (3) specifying the counterfactual intervention (e.g., no recommendation shown), and (4) reasoning about the counterfactual outcome (e.g., less likely to purchase a product online).
• Figure 2: (a) Causal graph structure and (b)(c) A factual/counterfactual example.
• Figure 3: Evaluation on causal graph construction. We evaluate F1 score to balance (i) whether the constructed edges under one category (e.g., $X\rightarrow M$) is correctly constructed if the ($X, Z, M, Y$) are already given. Additional results for all other datasets at Figure \ref{['fig:expt-task2-2']}.
• Figure 4: Evaluation of LLMs’ accuracy in identifying the correct intervention (i.e., the counterfactual value of $X$). Additional results for all other datasets are provided in Figure \ref{['fig:expt-task3-2']}.
• Figure 5: Additional evaluation on causal graph construction, complementing to Figure \ref{['fig:expt-task2']}.