Causal Cartographer: From Mapping to Reasoning Over Counterfactual Worlds
Gaël Gendron, Jože M. Rožanec, Michael Witbrock, Gillian Dobbie
TL;DR
The paper presents Causal Cartographer, a dual-framework that combines a graph retrieval-augmented causal extraction component with a step-by-step counterfactual reasoning agent to enable robust reasoning over counterfactual worlds. By constructing CausalWorld from real-world news (e.g., oil-related events), it creates a repository of causal knowledge with 975 nodes and 1337 edges to guide counterfactual inference under causal constraints. A dedicated CTG-Reason agent performs abduction-intervention-prediction in a way that respects causal structure, achieving competitive accuracy while greatly reducing context size and inference cost compared to standard chain-of-thought baselines. The approach demonstrates practical benefits for evaluating and improving LLMs on real-world counterfactual tasks and highlights potential societal impacts, including mitigations against bias and the importance of data reliability. Overall, the work advances causal reasoning in AI by linking explicit causal knowledge extraction with efficient, structured counterfactual inference.
Abstract
Causal world models are systems that can answer counterfactual questions about an environment of interest, i.e. predict how it would have evolved if an arbitrary subset of events had been realized differently. It requires understanding the underlying causes behind chains of events and conducting causal inference for arbitrary unseen distributions. So far, this task eludes foundation models, notably large language models (LLMs), which do not have demonstrated causal reasoning capabilities beyond the memorization of existing causal relationships. Furthermore, evaluating counterfactuals in real-world applications is challenging since only the factual world is observed, limiting evaluation to synthetic datasets. We address these problems by explicitly extracting and modeling causal relationships and propose the Causal Cartographer framework. First, we introduce a graph retrieval-augmented generation agent tasked to retrieve causal relationships from data. This approach allows us to construct a large network of real-world causal relationships that can serve as a repository of causal knowledge and build real-world counterfactuals. In addition, we create a counterfactual reasoning agent constrained by causal relationships to perform reliable step-by-step causal inference. We show that our approach can extract causal knowledge and improve the robustness of LLMs for causal reasoning tasks while reducing inference costs and spurious correlations.