Stress-Testing Causal Claims via Cardinality Repairs
Yarden Gabbay, Haoquan Guan, Shaull Almagor, El Kindi Rezig, Brit Youngmann, Babak Salimi
TL;DR
The paper tackles robustness of causal claims derived from observational data by introducing CaRET, a cardinality-repair framework. It presents SubCure with two scalable algorithms, SubCure-tuple and SubCure-pattern, and leverages incremental ATE updates to efficiently explore data edits without full retraining. Through extensive experiments on four real-world datasets and a synthetic benchmark, SubCure reveals compact, high-impact data regions whose removal can flip or reinforce causal conclusions, highlighting vulnerabilities often missed by traditional sensitivity analyses. The work offers a practical, interpretable tool for stress-testing causal analyses and guiding data cleaning, weighting, or targeted data collection.
Abstract
Causal analyses derived from observational data underpin high-stakes decisions in domains such as healthcare, public policy, and economics. Yet such conclusions can be surprisingly fragile: even minor data errors - duplicate records, or entry mistakes - may drastically alter causal relationships. This raises a fundamental question: how robust is a causal claim to small, targeted modifications in the data? Addressing this question is essential for ensuring the reliability, interpretability, and reproducibility of empirical findings. We introduce SubCure, a framework for robustness auditing via cardinality repairs. Given a causal query and a user-specified target range for the estimated effect, SubCure identifies a small set of tuples or subpopulations whose removal shifts the estimate into the desired range. This process not only quantifies the sensitivity of causal conclusions but also pinpoints the specific regions of the data that drive those conclusions. We formalize this problem under both tuple- and pattern-level deletion settings and show both are NP-complete. To scale to large datasets, we develop efficient algorithms that incorporate machine unlearning techniques to incrementally update causal estimates without retraining from scratch. We evaluate SubCure across four real-world datasets covering diverse application domains. In each case, it uncovers compact, high-impact subsets whose removal significantly shifts the causal conclusions, revealing vulnerabilities that traditional methods fail to detect. Our results demonstrate that cardinality repair is a powerful and general-purpose tool for stress-testing causal analyses and guarding against misleading claims rooted in ordinary data imperfections.