When Counterfactual Reasoning Fails: Chaos and Real-World Complexity

TL;DR

The paper investigates the reliability of counterfactual reasoning in dynamical systems under real-world conditions such as noise, parameter uncertainty, and chaos. By unifying state-space dynamics with structural causal modeling and using nested particle filtering for abduction, it constructs counterfactual trajectories (CF-SCM) and evaluates their reliability across Lorenz, Rössler, and logistic-growth systems. The findings show that even with accurate structure and measurements, slight parameter misspecification and chaotic dynamics can cause large divergences between generated counterfactuals and true trajectories, highlighting fundamental limits to counterfactual reasoning in complex systems. The work underscores the need for caution in applying counterfactual analyses to real-world, chaotic environments and points to future research avenues for more robust, validated counterfactual tools.

Abstract

Counterfactual reasoning, a cornerstone of human cognition and decision-making, is often seen as the 'holy grail' of causal learning, with applications ranging from interpreting machine learning models to promoting algorithmic fairness. While counterfactual reasoning has been extensively studied in contexts where the underlying causal model is well-defined, real-world causal modeling is often hindered by model and parameter uncertainty, observational noise, and chaotic behavior. The reliability of counterfactual analysis in such settings remains largely unexplored. In this work, we investigate the limitations of counterfactual reasoning within the framework of Structural Causal Models. Specifically, we empirically investigate \emph{counterfactual sequence estimation} and highlight cases where it becomes increasingly unreliable. We find that realistic assumptions, such as low degrees of model uncertainty or chaotic dynamics, can result in counterintuitive outcomes, including dramatic deviations between predicted and true counterfactual trajectories. This work urges caution when applying counterfactual reasoning in settings characterized by chaos and uncertainty. Furthermore, it raises the question of whether certain systems may pose fundamental limitations on the ability to answer counterfactual questions about their behavior.

Figures (9)

• Figure 1: Graphical representation that highlights the equivalence between SSMs and SCMs.
• Figure 2: Illustration of the proposed methodology to generate counterfactuals. Panel (a) shows the estimation of states and parameters, using the filtering/smoothing (NPFS) method, given a sequence of observations $\mathbf{Y}_{0:T}$. Panel (b) CF-SCM: After abducting the noise $\mathbf{U}_t^{\text{cf}}$, an intervention is applied to the initial state $\mathbf{X}_{0}$, resulting in a counterfactual trajectory $\mathbf{X}_{0:T}^{\text{cf}}$.
• Figure 3: Top row: Deterministic counterfactual trajectory (in red) compared to the generated counterfactual trajectories (in black) for the Lorenz system under three parameter assumptions: $\Tilde{\bm{\theta}} = {\bm{\theta}}_{true}$, $\Tilde{\bm{\theta}} = \hat{\bm{\theta}}$, and $\Tilde{\bm{\theta}} \sim \mathcal{N}(\hat{\bm{\theta}}, \bm{\sigma}_{\bm{\theta}})$. Bottom row: Observed sequence (black) alongside the estimated factual hidden sequence (blue), illustrating how even an accurate factual estimation may fail to produce reliable counterfactual trajectories once initial conditions are altered or parameter uncertainty is introduced.
• Figure 4: Top row: Deterministic counterfactual trajectory (in red) compared to the generated counterfactual trajectories (in black) for the Rössler system under three parameter assumptions: $\Tilde{\bm{\theta}} = {\bm{\theta}}_{true}$, $\Tilde{\bm{\theta}} = \hat{\bm{\theta}}$, and $\Tilde{\bm{\theta}} \sim \mathcal{N}(\hat{\bm{\theta}}, \bm{\sigma}_{\bm{\theta}})$. Bottom row: Observed sequence (black) alongside the estimated factual hidden sequence (blue), illustrating how even an accurate factual estimation may fail to produce reliable counterfactual trajectories once initial conditions are altered or parameter uncertainty is introduced.
• Figure 5: Top row: Deterministic counterfactual trajectory (in red) compared to the generated counterfactual trajectories (in black) for the logistic growth system under three parameter assumptions: $\Tilde{\bm{\theta}} = {\bm{\theta}}_{true}$, $\Tilde{\bm{\theta}} = \hat{\bm{\theta}}$, and $\Tilde{\bm{\theta}} \sim \mathcal{N}(\hat{\bm{\theta}}, \bm{\sigma}_{\bm{\theta}})$. Bottom row: Observed sequence (black) alongside the estimated factual hidden sequence (blue).