Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Invariance, Causality and Robustness

Peter Bühlmann

TL;DR

This work presents a unifying invariance-based approach to causal inference and predictive robustness under heterogeneous perturbations, connecting stable conditional distributions across environments to causal structure. It surveys Invariant Causal Prediction (ICP) and introduces anchor regression, including nonlinear variants and Anchor Boosting, with theory establishing a duality between worst-case risk and causal-regularized risk. The framework addresses invalid instruments, hidden confounding, and distributional robustness, offering practical algorithms, stopping rules, and variable-importance measures. Through applications in genomics and intervention-based prediction, the approach demonstrates improved predictive robustness and more interpretable causal guidance in the presence of environmental heterogeneity.

Abstract

We discuss recent work for causal inference and predictive robustness in a unifying way. The key idea relies on a notion of probabilistic invariance or stability: it opens up new insights for formulating causality as a certain risk minimization problem with a corresponding notion of robustness. The invariance itself can be estimated from general heterogeneous or perturbation data which frequently occur with nowadays data collection. The novel methodology is potentially useful in many applications, offering more robustness and better `causal-oriented' interpretation than machine learning or estimation in standard regression or classification frameworks.

Invariance, Causality and Robustness

TL;DR

This work presents a unifying invariance-based approach to causal inference and predictive robustness under heterogeneous perturbations, connecting stable conditional distributions across environments to causal structure. It surveys Invariant Causal Prediction (ICP) and introduces anchor regression, including nonlinear variants and Anchor Boosting, with theory establishing a duality between worst-case risk and causal-regularized risk. The framework addresses invalid instruments, hidden confounding, and distributional robustness, offering practical algorithms, stopping rules, and variable-importance measures. Through applications in genomics and intervention-based prediction, the approach demonstrates improved predictive robustness and more interpretable causal guidance in the presence of environmental heterogeneity.

Abstract

We discuss recent work for causal inference and predictive robustness in a unifying way. The key idea relies on a notion of probabilistic invariance or stability: it opens up new insights for formulating causality as a certain risk minimization problem with a corresponding notion of robustness. The invariance itself can be estimated from general heterogeneous or perturbation data which frequently occur with nowadays data collection. The novel methodology is potentially useful in many applications, offering more robustness and better `causal-oriented' interpretation than machine learning or estimation in standard regression or classification frameworks.

Paper Structure

This paper contains 35 sections, 6 theorems, 72 equations, 12 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. A framework based on invariance properties
  3. Our contribution
  4. Predicting potential outcomes, heterogeneity and worst case risk optimization
  5. Two examples for prediction of potential outcomes
  6. The heterogeneous setting with different environments
  7. A prediction problem and worst case risk optimization
  8. Invariance of conditional distributions
  9. Invariance and causality
  10. Invariant causal prediction
  11. Some robustness properties.
  12. Concrete tests.
  13. Application: single gene knock-out experiments.
  14. Unknown environments.
  15. Anchor regression: relaxing conditions
  16. ...and 20 more sections

Key Result

Proposition 1

Assume a partial structural equation model as in SEMY. Consider the set of environments ${\cal F}$ such that (B$({\cal F})$) holds. Then, the set of causal variables $S_{\mathrm{causal}} = \mathrm{pa}(Y)$ satisfies the invariance assumption with respect to ${\cal F}$, that is (A$_{S_{\mathrm{causal}

Figures (12)

  • Figure 1: Jerzy Neyman (1894-1981). Besides other pioneering work, he has also made fundamental early contributions to causality in 1923, in terms of mathematical formulation with the potential outcome model neyman23. Left: taken from http://www.learn-math.info/history/photos/Neyman_3.jpeg. Right: taken from https://errorstatistics.com/2017/04/16/a-spanos-jerzy-neyman-and-his-enduring-legacy-3/ by A. Spanos. The photograph is hanging on the wall in the coffee room of the Department of Statistics at UC Berkeley.
  • Figure 2: Graphical illustration of the ad-hoc conditions 1, 2 and the ad-hoc aim. There could be also hidden confounding variables, see Section \ref{['sec.anchor']}.
  • Figure 3: Trygve Haavelmo, Norwegian economist who received the Nobel Prize in Economic Sciences in 1989. Photo from https://en.wikipedia.org/wiki/Trygve_Haavelmo
  • Figure 4: Prediction of strong intervention effects in single gene deletion experiments in Saccharomyces cerevisiae (yeast). x-axis: number of predictions made by a method; y-axis: number of predictions being true strong intervention effects.. Invariant causal prediction (ICP) in red: the first 5 predictions are all true (and 7 among the first 9 predictions are true). Orange: the Causal Dantzig Selector rothenhausler2017, an algorithm based on an invariance property which includes hidden variables. All other methods are not distinguishable from random guessing (gray bars). Figure is taken from meetal16.
  • Figure 5: Graphical illustration with hidden confounding variables $H$. It corresponds to the instrumental variables regression model, where the instruments are now the environments.
  • ...and 7 more figures

Theorems & Definitions (6)

  • Proposition 1
  • Theorem 1
  • Proposition 2
  • Theorem 2
  • Proposition 3
  • Corollary 1