Valid Inference After Causal Discovery
Paula Gradu, Tijana Zrnic, Yixin Wang, Michael I. Jordan
TL;DR
This paper tackles the problem of obtaining valid statistical inferences for causal effects after causal discovery, addressing Freedman’s paradox caused by double dipping. It develops a randomized, differential-privacy–inspired framework that uses max-information bounds to correct downstream confidence intervals, ensuring coverage even when the causal graph is learned from the data. The main contributions include noisy versions of score-based search (noisy-select) and greedy search (noisy-ges) with finite-sample validity guarantees, theoretical connections to adaptive data analysis, and extensive empirical studies showing improved validity and competitive graph quality compared to data splitting. The approach is applicable to observational and interventional settings, scalable to high-dimensional problems, and robust to misspecification, offering a principled path for reliable causal inferences in practice.
Abstract
Causal discovery and causal effect estimation are two fundamental tasks in causal inference. While many methods have been developed for each task individually, statistical challenges arise when applying these methods jointly: estimating causal effects after running causal discovery algorithms on the same data leads to "double dipping," invalidating the coverage guarantees of classical confidence intervals. To this end, we develop tools for valid post-causal-discovery inference. Across empirical studies, we show that a naive combination of causal discovery and subsequent inference algorithms leads to highly inflated miscoverage rates; on the other hand, applying our method provides reliable coverage while achieving more accurate causal discovery than data splitting.