Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Causal Parrots: Large Language Models May Talk Causality But Are Not Causal

Matej Zečević, Moritz Willig, Devendra Singh Dhami, Kristian Kersting

TL;DR

This paper argues that large language models do not achieve true causality, proposing meta-Structural Causal Models (meta-SCMs) to capture causal facts about other SCMs and a Correlation of Causal Facts (CCF) hypothesis to explain occasional causal-looking behavior as data-driven recall rather than understanding. It formalizes the meta-SCM framework, and presents an extensive empirical program across common-sense reasoning, ground-truth causal discovery, and knowledge-base embeddings to test whether LLMs truly grasp causal structure or merely reflect training data correlations. Across multiple models and experimental setups, results show that while LLMs can sometimes reproduce correct causal facts, their inferences are noisy, highly prompt-dependent, and not reliable causal understanding, reinforcing the view that they are causal parrots rather than true causal learners. The work discusses methodological implications, limitations of current models, and ethical considerations for deploying such systems in safety-critical settings, while suggesting directions for leveraging LLMs as aiding tools rather than autonomous causal reasoners.

Abstract

Some argue scale is all what is needed to achieve AI, covering even causal models. We make it clear that large language models (LLMs) cannot be causal and give reason onto why sometimes we might feel otherwise. To this end, we define and exemplify a new subgroup of Structural Causal Model (SCM) that we call meta SCM which encode causal facts about other SCM within their variables. We conjecture that in the cases where LLM succeed in doing causal inference, underlying was a respective meta SCM that exposed correlations between causal facts in natural language on whose data the LLM was ultimately trained. If our hypothesis holds true, then this would imply that LLMs are like parrots in that they simply recite the causal knowledge embedded in the data. Our empirical analysis provides favoring evidence that current LLMs are even weak `causal parrots.'

Causal Parrots: Large Language Models May Talk Causality But Are Not Causal

TL;DR

This paper argues that large language models do not achieve true causality, proposing meta-Structural Causal Models (meta-SCMs) to capture causal facts about other SCMs and a Correlation of Causal Facts (CCF) hypothesis to explain occasional causal-looking behavior as data-driven recall rather than understanding. It formalizes the meta-SCM framework, and presents an extensive empirical program across common-sense reasoning, ground-truth causal discovery, and knowledge-base embeddings to test whether LLMs truly grasp causal structure or merely reflect training data correlations. Across multiple models and experimental setups, results show that while LLMs can sometimes reproduce correct causal facts, their inferences are noisy, highly prompt-dependent, and not reliable causal understanding, reinforcing the view that they are causal parrots rather than true causal learners. The work discusses methodological implications, limitations of current models, and ethical considerations for deploying such systems in safety-critical settings, while suggesting directions for leveraging LLMs as aiding tools rather than autonomous causal reasoners.

Abstract

Some argue scale is all what is needed to achieve AI, covering even causal models. We make it clear that large language models (LLMs) cannot be causal and give reason onto why sometimes we might feel otherwise. To this end, we define and exemplify a new subgroup of Structural Causal Model (SCM) that we call meta SCM which encode causal facts about other SCM within their variables. We conjecture that in the cases where LLM succeed in doing causal inference, underlying was a respective meta SCM that exposed correlations between causal facts in natural language on whose data the LLM was ultimately trained. If our hypothesis holds true, then this would imply that LLMs are like parrots in that they simply recite the causal knowledge embedded in the data. Our empirical analysis provides favoring evidence that current LLMs are even weak `causal parrots.'
Paper Structure (23 sections, 5 equations, 13 figures, 3 tables, 1 algorithm)

This paper contains 23 sections, 5 equations, 13 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Informal Summary of the Main Idea of the Paper
  3. Formalizing "Correlations of Causal Facts"
  4. Testing for Causal Knowledge in Large Language Models
  5. Methods and Results for "Common Sense" Inference Tasks
  6. Methods and Results for Causal Discovery on Ground Truth
  7. Method and Results for Knowledge Base Fact Embeddings
  8. Related Work
  9. Conclusive Discussion
  10. Takeaway, Ethical Challenges and Societal Implications.
  11. Appendix
  12. Another Philosophical Argument in Favor of Meta SCM: "Self-referencing Systems"
  13. Technical Details and Metrics
  14. Sentence templates
  15. Definition of Sparsity
  16. ...and 8 more sections

Figures (13)

  • Figure 1: Same Implication, Different Representations. When we consider the causal relationship between altitude (A) and temperature (T), then it is apparent that given the laws of physics we have an increase in altitude leading to a decrease in temperature. Graphically we can depict the relationship as A$\rightarrow$T, whereas the actual 'increase-decrease' relationship can only be specified through the SCM formalism with its structural equations, that is some $f$ such that $T=f(A,U)$ where $U$ are exogenous variables. The ground truth SCM underlying our laws of physics generates observational data in the form of numerical tuples $(a,t)$ as seen on the left scatter plot. To infer the casual relation, we can resort to algorithms for causal discovery. However, crucially, the same knowledge achieved through such induction can be represented within text for 'free' as one simply recites the Wikipedia article found on the right. While the article on the right is correct, and thus represents a fact about the actual world, there is no such guarantee for arbitrary other texts. That is, a model that simply obtains its knowledge from various Wikipedia statements will also learn untrue statements, statements that are not facts, thus explaining behavior that is correct sometimes and wrong other times.
  • Figure 2: Naïve Causal Discovery with LLMs.
  • Figure 3: Meta answers for unknown concepts.
  • Figure 4: Sensitivity to Query Wording.
  • Figure 5: Transfer of ConceptNet Causal Knowledge into Graph Predictions. Facts about driving influencing the fuel consumption can be found in the ConceptNet data (top). As a result the related edge "$\text{[D]riving style}\rightarrow\text{[F]uel consumption}$" of the driving data set gets predicted correctly in 4 out of 5 sentence wordings when applying k-NN classification. All templates match to the $\text{driving}\rightarrow\text{lack of fuel}$ ConceptNet fact as their nearest neighbor, except for the "Influence" template which matches to $\text{Moving car}\rightarrow\text{use fuel}$.
  • ...and 8 more figures

Theorems & Definitions (6)

  • Definition 1
  • example 1: 'Classical Setting'
  • Definition 2
  • Definition 3
  • example 2: 'Meta Setting'
  • Conjecture 1: Correlation of Causal Facts (CCF)