Causal Meta-Analysis: Rethinking the Foundations of Evidence-Based Medicine

TL;DR

This work reframes meta-analysis within a causal-inference framework by defining a target population $P^*$ as the convex mixture of study covariate distributions and proposing causal aggregation formulas. It shows that risk-difference meta-analyses admit a straightforward causal interpretation, while nonlinear measures like the risk ratio and odds ratio require new aggregation rules, derived via collapsibility concepts. Through theoretical development and a large-scale empirical comparison, the authors demonstrate potential mismatches between classical and causal meta-analysis, including real-world cases where conclusions diverge and policy implications follow. Finally, the article discusses privacy-preserving federated causal learning as a path forward to combine multiple data sources while respecting data confidentiality, outlining challenges and future directions for causal inference in evidence-based medicine.

Abstract

Meta-analysis, by synthesizing effect estimates from multiple studies conducted in diverse settings, stands at the top of the evidence hierarchy in clinical research. Yet, conventional approaches based on fixed- or random-effects models lack a causal framework, which may limit their interpretability and utility for public policy. Incorporating causal inference reframes meta-analysis as the estimation of well-defined causal effects on clearly specified populations, enabling a principled approach to handling study heterogeneity. We show that classical meta-analysis estimators have a clear causal interpretation when effects are measured as risk differences. However, this breaks down for nonlinear measures like the risk ratio and odds ratio. To address this, we introduce novel causal aggregation formulas that remain compatible with standard meta-analysis practices and do not require access to individual-level data. To evaluate real-world impact, we apply both classical and causal meta-analysis methods to 500 published meta-analyses. While the conclusions often align, notable discrepancies emerge, revealing cases where conventional methods may suggest a treatment is beneficial when, under a causal lens, it is in fact harmful.

Figures (4)

• Figure 1: The directed acyclic graph (DAG) representing the causal structure of our model. The variable $X$ is not (or partially) observed.
• Figure 2: Comparison of a classical meta-analysis with fixed or random effects with our causally interpretable approach on a synthetic dataset, for three different causal contrasts (risk difference, risk ratio, odds ratio). The plots for the risk ratio and the odds ratio are given in log-scale. The grey lines correspond to the null-effect value associated with the given causal contrast ($0$ for the risk difference, $1$ for the risk ratio and odds ratio) and the red line corresponds to our causally interpretable estimand $\theta(P^*)$.
• Figure 3: Comparison of a classical meta-analysis with random effects with our causally interpretable approach on the estimated effect of drug-eluting versus bare-metal stents in acute coronary syndrome.
• Figure 4: Comparison of a classical random-effects meta-analysis with our causally interpretable approach across 597 real-world meta-analyses.